{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3f187431",
   "metadata": {},
   "source": [
    "在此notebook中，我用一个简单的神经网络来训练MNIST数据集。训练过程仅调用了Numpy库，前向/反向传播的过程为手动实现。\n",
    "之后，我又依次添加了如下机制：\n",
    "- Dropout\n",
    "- Batch Gradient Descent\n",
    "- tanh和softmax激活函数\n",
    "\n",
    "误差收敛曲线利用matplotlib库绘制，详见各段代码下方。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f66a2726-1696-4d1d-ab09-d88363add642",
   "metadata": {},
   "source": [
    "# 反向传播过程手动实现\n",
    "手动实现神经网络反向传播，仅用到numpy库"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5000aa02",
   "metadata": {},
   "source": [
    "## 加载/预处理数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f01dd503-40ef-4c7a-9bee-15c24cf21fe8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys, numpy as np\n",
    "from keras.datasets import mnist\n",
    "\n",
    "# load the MNIST dataset\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "\n",
    "# 训练数据一维化，归一化\n",
    "images, labels = (x_train[0:1000].reshape(1000,28*28) / 255, y_train[0:1000])\n",
    "# 训练数据one-hot编码\n",
    "one_hot_labels = np.zeros((len(labels),10))\n",
    "for i,label in enumerate(labels):\n",
    "    one_hot_labels[i][label] = 1\n",
    "labels = one_hot_labels\n",
    "\n",
    "# 测试数据归一化\n",
    "test_images = x_test.reshape(len(x_test),28*28) / 255  \n",
    "# 测试数据one-hot编码\n",
    "test_labels = np.zeros((len(y_test),10))\n",
    "for i,label in enumerate(y_test):\n",
    "    test_labels[i][label] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aae903bd",
   "metadata": {},
   "source": [
    "## 初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6d0a25c0-3d9e-4c4d-8c43-ac3b459d57c3",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(1)\n",
    "\n",
    "def relu(x):\n",
    "    return (x >= 0) * x  # ReLU激活函数\n",
    "\n",
    "def relu2deriv(output):\n",
    "    return output>=0  # ReLU函数的导数\n",
    "\n",
    "alpha, epochs, hidden_size, pixels_per_image, num_labels = (0.005, 400, 40, 784, 10)\n",
    "\n",
    "# 权重初始化\n",
    "weights_0_1 = 0.2*np.random.random((pixels_per_image,hidden_size)) - 0.1\n",
    "weights_1_2 = 0.2*np.random.random((hidden_size,num_labels)) - 0.1\n",
    "\n",
    "# 可视化数据记录\n",
    "x_axis_epoch = []\n",
    "y_axis_train_error = []\n",
    "y_axis_test_error = []"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95c672c8",
   "metadata": {},
   "source": [
    "## 模型训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1aebf1b9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I:0 Train-Err:0.722 Train-Acc:0.537 Test-Err:0.601 Test-Acc:0.6488\n",
      "I:10 Train-Err:0.312 Train-Acc:0.901 Test-Err:0.420 Test-Acc:0.8114\n",
      "I:20 Train-Err:0.260 Train-Acc:0.93 Test-Err:0.414 Test-Acc:0.8111\n",
      "I:30 Train-Err:0.232 Train-Acc:0.946 Test-Err:0.417 Test-Acc:0.8066\n",
      "I:40 Train-Err:0.215 Train-Acc:0.956 Test-Err:0.426 Test-Acc:0.8019\n",
      "I:50 Train-Err:0.204 Train-Acc:0.966 Test-Err:0.437 Test-Acc:0.7982\n",
      "I:60 Train-Err:0.194 Train-Acc:0.967 Test-Err:0.448 Test-Acc:0.7921\n",
      "I:70 Train-Err:0.186 Train-Acc:0.975 Test-Err:0.458 Test-Acc:0.7864\n",
      "I:80 Train-Err:0.179 Train-Acc:0.979 Test-Err:0.466 Test-Acc:0.7817\n",
      "I:90 Train-Err:0.172 Train-Acc:0.981 Test-Err:0.474 Test-Acc:0.7758\n",
      "I:100 Train-Err:0.166 Train-Acc:0.984 Test-Err:0.482 Test-Acc:0.7706\n",
      "I:110 Train-Err:0.161 Train-Acc:0.984 Test-Err:0.489 Test-Acc:0.7686\n",
      "I:120 Train-Err:0.157 Train-Acc:0.986 Test-Err:0.496 Test-Acc:0.766\n",
      "I:130 Train-Err:0.153 Train-Acc:0.99 Test-Err:0.502 Test-Acc:0.7622\n",
      "I:140 Train-Err:0.149 Train-Acc:0.991 Test-Err:0.508 Test-Acc:0.758\n",
      "I:150 Train-Err:0.145 Train-Acc:0.991 Test-Err:0.513 Test-Acc:0.7558\n",
      "I:160 Train-Err:0.141 Train-Acc:0.992 Test-Err:0.518 Test-Acc:0.7553\n",
      "I:170 Train-Err:0.138 Train-Acc:0.992 Test-Err:0.524 Test-Acc:0.751\n",
      "I:180 Train-Err:0.135 Train-Acc:0.995 Test-Err:0.528 Test-Acc:0.7505\n",
      "I:190 Train-Err:0.132 Train-Acc:0.995 Test-Err:0.533 Test-Acc:0.7482\n",
      "I:200 Train-Err:0.130 Train-Acc:0.998 Test-Err:0.538 Test-Acc:0.7464\n",
      "I:210 Train-Err:0.127 Train-Acc:0.998 Test-Err:0.544 Test-Acc:0.7446\n",
      "I:220 Train-Err:0.125 Train-Acc:0.998 Test-Err:0.552 Test-Acc:0.7416\n",
      "I:230 Train-Err:0.123 Train-Acc:0.998 Test-Err:0.560 Test-Acc:0.7372\n",
      "I:240 Train-Err:0.121 Train-Acc:0.998 Test-Err:0.569 Test-Acc:0.7344\n",
      "I:250 Train-Err:0.120 Train-Acc:0.999 Test-Err:0.577 Test-Acc:0.7316\n",
      "I:260 Train-Err:0.118 Train-Acc:0.999 Test-Err:0.585 Test-Acc:0.729\n",
      "I:270 Train-Err:0.117 Train-Acc:0.999 Test-Err:0.593 Test-Acc:0.7259\n",
      "I:280 Train-Err:0.115 Train-Acc:0.999 Test-Err:0.600 Test-Acc:0.723\n",
      "I:290 Train-Err:0.114 Train-Acc:0.999 Test-Err:0.607 Test-Acc:0.7196\n",
      "I:300 Train-Err:0.113 Train-Acc:0.999 Test-Err:0.614 Test-Acc:0.7183\n",
      "I:310 Train-Err:0.112 Train-Acc:0.999 Test-Err:0.622 Test-Acc:0.7165\n",
      "I:320 Train-Err:0.111 Train-Acc:0.999 Test-Err:0.629 Test-Acc:0.7133\n",
      "I:330 Train-Err:0.110 Train-Acc:0.999 Test-Err:0.637 Test-Acc:0.7125\n",
      "I:340 Train-Err:0.109 Train-Acc:1.0 Test-Err:0.645 Test-Acc:0.71\n",
      "I:350 Train-Err:0.108 Train-Acc:1.0 Test-Err:0.654 Test-Acc:0.7073\n",
      "I:360 Train-Err:0.108 Train-Acc:0.999 Test-Err:0.662 Test-Acc:0.704\n",
      "I:370 Train-Err:0.107 Train-Acc:0.999 Test-Err:0.671 Test-Acc:0.702\n",
      "I:380 Train-Err:0.107 Train-Acc:0.998 Test-Err:0.679 Test-Acc:0.7007\n",
      "I:390 Train-Err:0.106 Train-Acc:0.998 Test-Err:0.685 Test-Acc:0.6985\n",
      "I:399 Train-Err:0.106 Train-Acc:0.998 Test-Err:0.691 Test-Acc:0.6971\n"
     ]
    }
   ],
   "source": [
    "for j in range(epochs):\n",
    "    error, correct_cnt = (0.0, 0)  # 训练集误差，预测正确个数计数\n",
    "    \n",
    "    for i in range(len(images)):  # SGD\n",
    "        # 前向传播\n",
    "        layer_0 = images[i:i+1]\n",
    "        layer_1 = relu(layer_0.dot(weights_0_1))\n",
    "        layer_2 = layer_1.dot(weights_1_2)\n",
    "\n",
    "        error += np.sum((labels[i:i+1] - layer_2) ** 2)  # 均方差\n",
    "        correct_cnt += int(np.argmax(layer_2) == np.argmax(labels[i:i+1])) # 预测正确个数计数\n",
    "\n",
    "        # 反向传播\n",
    "        layer_2_delta = (labels[i:i+1] - layer_2)\n",
    "        layer_1_delta = layer_2_delta.dot(weights_1_2.T) * relu2deriv(layer_1)\n",
    "        \n",
    "        # 更新权重\n",
    "        weights_1_2 += alpha * layer_1.T.dot(layer_2_delta)\n",
    "        weights_0_1 += alpha * layer_0.T.dot(layer_1_delta)\n",
    "\n",
    "    # 每隔10次迭代输出结果，并记录测试集准确率\n",
    "    if(j % 10 == 0 or j == epochs-1):\n",
    "        \n",
    "        # 测试集\n",
    "        test_error, test_correct_cnt = (0.0, 0)   # 测试集误差，预测正确个数计数\n",
    "\n",
    "        for i in range(len(test_images)):\n",
    "\n",
    "            layer_0 = test_images[i:i+1]\n",
    "            layer_1 = relu(layer_0.dot(weights_0_1))\n",
    "            layer_2 = layer_1.dot(weights_1_2)\n",
    "            \n",
    "            test_error += np.sum((test_labels[i:i+1] - layer_2) ** 2)\n",
    "            test_correct_cnt += int(np.argmax(layer_2) == \\\n",
    "                                            np.argmax(test_labels[i:i+1]))\n",
    "            \n",
    "        sys.stdout.write(\"I:\"+str(j)+ \\\n",
    "                     \" Train-Err:\" + str(error/float(len(images)))[0:5] +\\\n",
    "                     \" Train-Acc:\" + str(correct_cnt/float(len(images))))\n",
    "        sys.stdout.write(\" Test-Err:\" + str(test_error/float(len(test_images)))[0:5] +\\\n",
    "                         \" Test-Acc:\" + str(test_correct_cnt/float(len(test_images))))\n",
    "        print()  # 换行\n",
    "        \n",
    "        x_axis_epoch.append(j) # 记录轮次\n",
    "        y_axis_train_error.append(error/float(len(images))) # 记录训练集误差\n",
    "        y_axis_test_error.append(test_error/float(len(test_images))) # 记录测试集误差\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cc2bfe2",
   "metadata": {},
   "source": [
    "## 训练过程可视化\n",
    "训练误差不断下降，测试误差先下降后不断上升，说明模型过拟合。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "0cffbc7f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcoAAAE6CAYAAACWDhLFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABX50lEQVR4nO3deVxU9f748dcMDPsmouCCiLuIWoolmnugkGa2+WsxTbtlpDezTbNyyZvd6pr1vWl5b2raLc200vSqZLmktzJTc8s0F1RAFBdABAbm/P44zsAwbMIMZwbez8fjPGbmc5Z5z5nRN+dzPotOURQFIYQQQpRJr3UAQgghhDOTRCmEEEJUQBKlEEIIUQFJlEIIIUQFJFEKIYQQFZBEKYQQQlRAEqUQQghRAUmUQgghRAUkUQohhBAVkEQpHEqn01Vp2bJlS43eZ8aMGeh0OvsEfV1F8Y4ZM8au7+WKKjvnW7ZsqfL3bw+HDh1ixowZnDx5skrbL1myxKG/SVF3uGsdgKjb/ve//1m9fu211/j+++/57rvvrMqjoqJq9D6PPfYYQ4YMqdExynLvvffy7LPP2pQ3atTI7u9V13Tr1s3m+x8xYgStW7fm7bfftvv7HTp0iJkzZ9K/f39atmxZ5f0WL15Mhw4dbMpr+psUdYckSuFQPXv2tHrdqFEj9Hq9TXlpubm5+Pj4VPl9mjdvTvPmzasVY0VCQ0MrjbUs5cWvKAp5eXl4e3tXO6Zr167h5eVl9ytoewsICLA5d56engQFBVXrnDpKdHQ0MTExN7RPRd+jPb6fG/39C8eSqlehuf79+xMdHc22bdvo1asXPj4+jB07FoAVK1YQHx9PkyZN8Pb2pmPHjkyZMoWrV69aHaOsasCWLVsydOhQNmzYQLdu3fD29qZDhw4sWrTIrvGPGTMGPz8/9u/fT3x8PP7+/gwaNAhQq28nTJjABx98QMeOHfH09OTjjz8G4IcffmDQoEH4+/vj4+NDr169WLdundWxzdWDmzZtYuzYsTRq1AgfHx/y8/PLjCUvL49nn32Wm266icDAQIKDg4mNjeXrr7+22dYc27Jly+jYsSM+Pj507dqVb775xmbbdevWcdNNN+Hp6UlkZKRdrwjT09N54oknaN68OR4eHkRGRjJz5kwKCwuttluwYAFdu3bFz88Pf39/OnTowEsvvQSo5+m+++4DYMCAAZbq0yVLltglxvK+x4q+H5PJxJtvvkmHDh3w9PSkcePGPPLII5w5c8bq2BX9/oVzkCtK4RTS0tJ4+OGHeeGFF3j99dfR69W/4Y4ePUpiYiKTJk3C19eX33//nb///e/8/PPPNtW3Zdm3bx/PPvssU6ZMITQ0lH//+9+MGzeONm3a0Ldv30r3VxTF5j9sADc3N6vEXFBQwJ133skTTzzBlClTrPb56quv2L59O6+++iphYWE0btyYrVu3EhcXR5cuXfjoo4/w9PRk/vz5DBs2jM8++4yRI0davd/YsWO54447WLZsGVevXsVgMJQZb35+PhcvXuS5556jWbNmFBQU8O2333L33XezePFiHnnkEavt161bx65du5g1axZ+fn68+eabjBgxgiNHjtCqVSsANm/ezPDhw4mNjWX58uUUFRXx5ptvcu7cuUrPX2XS09O55ZZb0Ov1vPrqq7Ru3Zr//e9/zJ49m5MnT7J48WIAli9fTlJSEhMnTuTtt99Gr9dz7NgxDh06BMAdd9zB66+/zksvvcT7779Pt27dAGjdunWlMRQVFdl8xzqdDjc3N6uysr7HXbt2AWV/P08++SQLFy5kwoQJDB06lJMnT/LKK6+wZcsWfv31V0JCQizHLu/3L5yEIkQtGj16tOLr62tV1q9fPwVQNm/eXOG+JpNJMRqNytatWxVA2bdvn2Xd9OnTldI/54iICMXLy0s5deqUpezatWtKcHCw8sQTT1QaK1DusmzZMqvPBCiLFi0q8xiBgYHKxYsXrcp79uypNG7cWMnOzraUFRYWKtHR0Urz5s0Vk8mkKIqiLF68WAGURx55pNJ4y1JYWKgYjUZl3Lhxys0332wTW2hoqJKVlWUpS09PV/R6vTJnzhxL2a233qo0bdpUuXbtmqUsKytLCQ4OtjnnlYmIiFDuuOMOy+snnnhC8fPzs/qOFEVR3n77bQVQDh48qCiKokyYMEEJCgqq8NgrV65UAOX777+vUizmc1vW4ubmZrVted9jed/P4cOHFUBJSkqyKv/pp58UQHnppZcsZVX9/QvtyJ8twik0aNCAgQMH2pQfP36cBx98kLCwMNzc3DAYDPTr1w+Aw4cPV3rcm266iRYtWlhee3l50a5dO06dOlWluO6//3527dplsyQmJtpse88995R5jIEDB9KgQQPL66tXr/LTTz9x77334ufnZyl3c3Nj1KhRnDlzhiNHjlTp2GVZuXIlvXv3xs/PD3d3dwwGAx999FGZ52vAgAH4+/tbXoeGhtK4cWPL+bl69Sq7du3i7rvvxsvLy7Kdv78/w4YNq3JM5fnmm28YMGAATZs2pbCw0LIkJCQAsHXrVgBuueUWLl++zAMPPMDXX3/NhQsXavzeZkuXLrX5fn/66Seb7Up/jyWV/n6+//57AJvW0bfccgsdO3Zk8+bNVuXl/f6Fc5CqV+EUmjRpYlOWk5NDnz598PLyYvbs2bRr1w4fHx9Onz7N3XffzbVr1yo9bsOGDW3KPD09q7QvqI2PqtLQw8fHh4CAgDLXlf5sly5dQlGUMj9z06ZNAcjMzKzwGOVZvXo1999/P/fddx/PP/88YWFhuLu7s2DBgjLvzVZ2fi5duoTJZCIsLMxmu7LKbtS5c+dYu3ZtuVXJ5oQ4atQoCgsL+de//sU999yDyWSiR48ezJ49m7i4uBrF0LFjxyp9xxV9B6XXmb+/8r7j0n+oVfX7FdqQRCmcQlktBL/77jtSU1PZsmWL5SoS4PLly7UYWdVU1MKx9LoGDRqg1+tJS0uz2TY1NRXA6v5VZccv6ZNPPiEyMpIVK1ZY7VNe45/KNGjQAJ1OR3p6us26sspuVEhICF26dOFvf/tbmevNfzgAPProozz66KNcvXqVbdu2MX36dIYOHcoff/xBREREjWOpzI18x+Y/QNLS0mxaY6emplb7+xXakKpX4bTM/3l4enpalX/44YdahGM3vr6+3HrrraxevdrqytZkMvHJJ5/QvHlz2rVrV61j63Q6PDw8rP7jTU9PL7PVa1VjveWWW1i9ejV5eXmW8uzsbNauXVutY5Y0dOhQDhw4QOvWrYmJibFZSibKkjElJCQwbdo0CgoKOHjwIFD8O6lqbYEjmatRP/nkE6vyXbt2cfjwYUuraOEa5IpSOK1evXrRoEEDxo8fz/Tp0zEYDPznP/9h3759tRbDuXPn+PHHH23KAwICatQhfc6cOcTFxTFgwACee+45PDw8mD9/PgcOHOCzzz6r9hXG0KFDWb16NUlJSdx7772cPn2a1157jSZNmnD06NFqHfO1115jyJAhxMXF8eyzz1JUVMTf//53fH19uXjxYrWOaTZr1iySk5Pp1asXf/3rX2nfvj15eXmcPHmS9evX88EHH9C8eXP+8pe/4O3tTe/evWnSpAnp6enMmTOHwMBAevToAaj9IQEWLlyIv78/Xl5eREZGllm9XNKBAwfKbNncunXrag8s0b59ex5//HH+7//+D71eT0JCgqXVa3h4OM8880y1jiu0IYlSOK2GDRuybt06nn32WR5++GF8fX0ZPnw4K1assDT/d7QvvviCL774wqa8d+/e/PDDD9U+br9+/fjuu++YPn06Y8aMwWQy0bVrV9asWcPQoUOrfdxHH32UjIwMPvjgAxYtWkSrVq2YMmUKZ86cYebMmdU6ZlxcHF999RUvv/wyI0eOJCwsjKSkJK5du1btY5o1adKEX375hddee4233nqLM2fO4O/vT2RkJEOGDLE0nunTpw9Llizh888/59KlS4SEhHDbbbexdOlSSzKLjIxk3rx5vPvuu/Tv35+ioiIWL15c6XCDjz76aJnl//rXv3jssceq/dkWLFhA69at+eijj3j//fcJDAxkyJAhzJkzp9LkLZyLTlEUResghBBCCGcl9yiFEEKICkiiFEIIISogiVIIIYSogCRKIYQQogKSKIUQQogKSKIUQgghKlDv+lGaTCZSU1Px9/eXYaOEEKIeUxSF7OxsmjZtWuHUZvUuUaamphIeHq51GEIIIZzE6dOnbcbkLaneJUrzlEKnT58ud7aHqjAajWzatIn4+PhyZz5wJhKvY7lavOB6MUu8jlUf483KyiI8PNxqqrmy1LtEaa5uDQgIqHGiNE+t5Co/KonXcVwtXnC9mCVex6rP8VZ2G04a8wghhBAVkEQphBBCVEASpRBCCFEBSZRCCCFEBSRRCiGEEBWQRCmEEEJUoN51D7GH7GwYONCN1NRBHD0KLtCSWgghRDVJoqwGb2/45Rc94MeVK0Z8fLSOSAghhKNI1Ws1uLuDn58CwOXL2sYihBDCsSRRVlNQkPp45YoMrC6EEHWZJMpqCgxUH+WKUggh6jbNE+X8+fOJjIzEy8uL7t27s3379nK3HTNmDDqdzmbp1KlTLUasCgpSq14vXar1txZCCFGLNE2UK1asYNKkSUybNo09e/bQp08fEhISSElJKXP7d999l7S0NMty+vRpgoODue+++2o58uIryitXav2thRBC1CJNE+XcuXMZN24cjz32GB07dmTevHmEh4ezYMGCMrcPDAwkLCzMsvzyyy9cunSJRx99tJYjhwYN1MfLl+UepRBC1GWadQ8pKChg9+7dTJkyxao8Pj6enTt3VukYH330EbfffjsRERHlbpOfn09+fr7ldVZWFqBO0WI0GqsRuUqdoUvPxYsmjEZTtY9TW8yftSafuTZJvI7najFLvI5VH+Ot6r6aJcoLFy5QVFREaGioVXloaCjp6emV7p+WlsZ///tfPv300wq3mzNnDjNnzrQp37RpEz416ACZmdkBaM+BA2dYv/63ah+ntiUnJ2sdwg2ReB3P1WKWeB2rPsWbm5tbpe00H3Cg9ISZiqJUOokmwJIlSwgKCuKuu+6qcLupU6cyefJky2vzjNbx8fE1mrj58GGFzz8Hf/9wEhObV/s4tcVoNJKcnExcXJzLTMoq8TqWq8Us8TqWy8RrNKLbuxfTli18Gx5O/3vuqXa85hrGymiWKENCQnBzc7O5eszIyLC5yixNURQWLVrEqFGj8PDwqHBbT09PPD09bcoNBkONfgwNGxYCkJ2tx2DQvPFwldX0c9c2idfxXC1midexnC7ea9fg559h2zZ12bkTrl8JhkyZguH//b9qx1vV/TRLlB4eHnTv3p3k5GRGjBhhKU9OTmb48OEV7rt161aOHTvGuHHjHB1mucytXqV7iBBC2FFuLuzYAVu2qInx55+hoMB6m+BgTL17Y/T1rZWQNK16nTx5MqNGjSImJobY2FgWLlxISkoK48ePB9Rq07Nnz7J06VKr/T766CNuvfVWoqOjtQgbkFavQghhF0ajmgy/+w42b4b//c82MYaFQb9+0LevukRFUVRUxIX162slRE0T5ciRI8nMzGTWrFmkpaURHR3N+vXrLa1Y09LSbPpUXrlyhVWrVvHuu+9qEbJFYKByPR5NwxBCCNdiMsG+fcWJcds2uHrVeptmzWDgQDU59usHrVtD6bYrRUW1FrLmjXmSkpJISkoqc92SJUtsygIDA6vcUsmRzGO9yhB2QghRiQsXYNMm2LABNm6EjAzr9Q0bwoABMGiQmiDbtrVNjBrSPFG6KnOivHZNR34+lNFeSAgh6qeiIrU6dcMGddm1CxSleL2fn3qlOHCgmhw7dwa98zaKlERZTSV7lly5Ao0baxeLEEJoLjcX1q+HL75Qrx5Lt3Ts0gWGDIGEBOjVCyrpseBMJFFWk5sb+PgYyc01cOmSJEohRD2Ul6dWpa5YAWvWWN9rDAqCuDg1MQ4eDE2bahZmTUmirAFfXzVRyn1KIUS9UVAAyclqcvz6ayjZaT8iAu6/H+66C265RZ3lvg6oG59CI76+Rs6flwY9Qog6zmhUW6l+/jl8+aV1tWqzZmpyvP9+uPVWp2qEYy+SKGvA11cdUFcSpRCizjEnx5Ur1eR48WLxutBQuO8+GDlSvd/oxA1x7EESZQ1IohRC1CW6wkJ0ycmwerVtcmzcGO6+W71y7NtXbahRT0iirAFJlEIIl1dYCFu24PbZZwxZuRL37OzidY0awT33qFePffvWmXuON6p+fmo7MSdKGe9VCOFSiorUEXFWrIBVq+DCBfSAB6A0aoROkqMVOQM14OcnV5RCCBdRVKQONm5OjufOFa9r2JCiESP4MTycW55/HoO3t3ZxOiFJlDUgVa9CCKdWWAjbt6v3HFetgrS04nXBwcX3HAcMwKQo6iDjcgVpQ85IDUiiFEI4nbw8+PZbNTmuWQOZmcXrgoJgxAg1OQ4aBCXnYzQaaz1UVyGJsgYkUQohnEJ2tjp83OrV6mNOTvG6kBAYPly9erz9dpcaOs5ZSKKsAUmUQgjNnD4Na9eqy3ffWc/h2KyZmhjvvhtuu02qU2tIzl4NSKIUQtQakwl++aU4Oe7bZ72+bVu1K8eIERATU+cHAahNkihroGT3EEWpkyM3CSG0dPWqer9x7VpYtw7S04vX6XQQGwvDhqlLVJT8J+QgkihrwNw9pKBAvX8uLaqFEDWiKPD77/Df/6rLtm3WVap+fupMHMOGQWKiOiCAcDhJlDXg5VWIXq9gMum4fFkSpRCiGq5eVe8xmpPjyZPW61u2hKFD1eTYr5/MEq8BSZQ1oNdDYKBa9Xr5MjRponVEQginpyhw8KA6j+PGjbB1q/VVo4eHmhATEtSlfXupUtWYJMoaCgoqTpRCCFGmzEz1XuPGjbBpE5w9a72+ZUu1KjUhAQYMAF9fTcIUZZNEWUOBgeqjJEohhEVhIbqdO+nw6ae4/e1vamtVRSle7+UF/fur9xuHDJGrRicnibKGgoIUQCcDowtR3506VVydunkz7leu0L7k+k6d1KQ4eDD06aMmS+ESJFHWUFCQ+ihXlELUM1evwpYtalXqxo1w5IjVaqVBA1KjoggdPRr3hARo3lybOEWNad4jdf78+URGRuLl5UX37t3Zvn17hdvn5+czbdo0IiIi8PT0pHXr1ixatKiWorUliVKIekJR4MABePttdSi44GC1Nep776lJ0s0NeveGWbPgp58oTE3ll+efRxkzRpKki9P0inLFihVMmjSJ+fPn07t3bz788EMSEhI4dOgQLVq0KHOf+++/n3PnzvHRRx/Rpk0bMjIyKCwsrOXIr1OU61WvkiiFqJMuX1Yb4WzYoC5lNcIZPBji42HgwOK/nEEGGa9DNE2Uc+fOZdy4cTz22GMAzJs3j40bN7JgwQLmzJljs/2GDRvYunUrx48fJzg4GICWLVvWZsiq8+dxj4tjyMmT7H36POAmiVKIusBkgl9/LU6MP/6ozuNo5u2tNsIx32ts104a4dQDmiXKgoICdu/ezZQpU6zK4+Pj2blzZ5n7rFmzhpiYGN58802WLVuGr68vd955J6+99hre5fT2z8/PJz8/3/I6KysLAKPRiLG6f/H5+eF+8CCehYU0KToFtOHiRRNGY1Glu2rF/Fmr/ZlrmcTreK4Ws8PivXABXXIy+o0b0SUnozt/3mq10qEDpsGDUeLjUW67zXpkkQpqs+T8OpY94q3qvpolygsXLlBUVERoaKhVeWhoKOklxzMs4fjx4/zwww94eXnx5ZdfcuHCBZKSkrh48WK59ynnzJnDzJkzbco3bdqEj49PteMf1LgxfqmpGA9/C7Th2LELrF//v2ofr7YkJydrHcINkXgdz9VirnG8RUU0OHaMxr/+SuivvxJ07Bi6El03jN7eXOjShXPdupFx881ca9z4+gojfP997cdby+pTvLm5uVXaTvNWr7pS1RaKotiUmZlMJnQ6Hf/5z38IvN6Bce7cudx77728//77ZV5VTp06lcmTJ1teZ2VlER4eTnx8PAEBAdWOW9+5M6SmEhui/kXp7h5CYmJitY/naEajkeTkZOLi4jCUnKzVSUm8judqMdco3osX0W3ciP6//1WvGktOZgwoXbpgio9HGTIEevYkxMODEKCTVvFqoD7Ga65hrIxmiTIkJAQ3Nzebq8eMjAybq0yzJk2a0KxZM0uSBOjYsSOKonDmzBnatm1rs4+npyeeZYyNaDAYavRjKGrXDjZuJOTKCQCuXNFjMGjeiLhSNf3ctU3idTxXi7lK8SoK7N+vzrixbh3873/q/UezoCCIi1NHwhk8GF3TprhpGa8TqU/xVnU/zRKlh4cH3bt3Jzk5mREjRljKk5OTGT58eJn79O7dm5UrV5KTk4Ofnx8Af/zxB3q9nua13fy6dWsAAs4dBaTVqxCay81VBxc3J8fTp63Xd+4Md9yhLj17ymTGoso0/aVMnjyZUaNGERMTQ2xsLAsXLiQlJYXx48cDarXp2bNnWbp0KQAPPvggr732Go8++igzZ87kwoULPP/884wdO7bcxjyOorRpA4BP6p+AmihlTkohatnZs2pSXLtW7caRl1e8ztsbBg1SE2NiIpTT5UyIymiaKEeOHElmZiazZs0iLS2N6Oho1q9fT0REBABpaWmkpKRYtvfz8yM5OZmJEycSExNDw4YNuf/++5k9e3atx65cv6I0pBxHh4nCQj25uTKWsRAOZTKh271b7bqxdq3alaOkFi3UQQDuuEMdXFzmvhN2oHndQ1JSEklJSWWuW7JkiU1Zhw4dnKNVVkQEJjc39Hl5tNCf5ZQpnMuXJVEKYXf5+bB5M26rVzN49WrcSw6srNPBrbeqczUOGwbR0VKtI+xO80TpstzdyQ0NxS81lZv8jnEqK5xLl6BZM60DE6IOyMqC9evhq6/Ux+xs9IAXoPj5oYuPVxNjYiKYu28I4SCSKGsgp0kT/FJTifY8ytcMkAY9QtTEuXOwZg18+SVs3mw9mXGTJhTdeSc/hYXR47nnMFxvzCdEbZBEWQNXmzQBoJ3+GCAtX4W4YenpsHIlfP457NhhPWdju3YwYoS69OiBqaiI8+vXQxndvYRwJEmUNWBOlK1M0kVEiCo7fx5WrYIVK2DrVuvkGBNTnBw7drTer8h5h4gUdZskyhowJ8oW+XJFKUSFLl5Uq1RXrFD7OpZMerfeCiNHwr33Qni4djEKUQ5JlDVgTpRhV4+hw8Tly84/Mo8QtebiRfj6a7VqNTnZegDx7t3V5HjffepUVUI4MUmUNZDbuDGKuzsehXk0JZVLl2RyVlHPZWaqLVVXrlQb5JRMjl26qMnx/vvh+oAdQrgCSZQ1oLi5qX8NHztGW45y+bIkSlEPXbigVqt+8YWaHEtWq3burF413nuv7T1HIVyEJMoaUtq0QXfsGG04xsXLA7QOR4ja8eef6sg4a9bAtm3WybFr1+Lk2L69djEKYSeSKGvIPJRdW46y8bK2sQjhMEVF8OOPanJcuxYOHbJef9NNxcmxXTtNQhTCUSRR1tT1ey1tOMaKy9qGIoRdXbqkTlS8Zo068PiFC8Xr3Nygb1+48051hJzrfzAKURdJoqyhkleU0j1EuLTz59Vq1G3b1P6Nv/1m3ccxMFAdMm7YMBgyBBo00C5WIWqRJMoaMk+31Zo/uXzRBEgXEeEizp6l2fbt6Nevh+3b4fBh223atVOT4513wm23gQtN6CuEvUiirKmICBQ3N3yKruF7JRWTqTl6yZXCmSgKnDkDu3er01JdXwxpacSU3jY6Wq1S7dcP+vSB632FhajPJFHWlMGA0jIS3Z/HaKUcIyenOQEBWgcl6i2jEY4ehQMHYM+e4sRY8v7idYpez5WWLfEfOhS3AQPUK8aQEA2CFsK5SaK0A13bNvCnuS9lf0mUwvGKiuD4cTUhHjxY/HjkiJosS3N3h06doFs3y1IYFcXWrVtJTEzETapUhSiXJEo70LVtCxs20IZjXL6sTrIuhF0oCpw9qybC/fvV5cAB9X5iXl7Z+/j5QVSU2mXDnBg7dwYvL+vtykqoQggbkijt4XqDHmn5Kmrk6lXYuxf27StOjAcOlD/avre3OtpNp07qvUXzY3g4cqNcCPuRRGkPJfpSnrysbSjCReTkqElx9+7i5fffwWSy3dbNTR3hJjpavTKMjlaXyEh1nRDCoSRR2kPbtoCaKPdcVACdtvEI55KXpybFn3+GXbuKk2LJPopmTZoUV5WaE2P79jJZsRAakkRpDy1bUqRzw0e5hvFUKtBM64iEVoqK1AY15qT4889qx/2Ss2iYNW2qTjdVcpHuGEI4HUmU9mAwkOnfksZZf+J+8hiSKOsJRYGTJ2HXLvQ//kjvTZtwf+gh9V5jaY0bQ48e6hIToybFsLBaD1kIceMkUdrJpYZtaJz1J15njgL9tA5HOEJ6unqVWHLJzATADbD0QPTzUxPhLbeoS48ealNonVTJC+GKNE+U8+fP56233iItLY1OnToxb948+vTpU+a2W7ZsYcAA26msDh8+TIcOHRwdaoVywtrCiY34px/TNA5hJwUF6n3FnTvhf/9TZ85ISbHdzmCArl0p6t6d3zw8iB47FkPnztLIRog6RNNEuWLFCiZNmsT8+fPp3bs3H374IQkJCRw6dIgWFXRGPHLkCAElevU3atSoNsKtUF5zteVrcOZRjSMR1XLunJoQzYnxl19s+ynqdGp3DHMVao8e6tyLnp6YjEZS1q8nulMnSZJC1DGaJsq5c+cybtw4HnvsMQDmzZvHxo0bWbBgAXPmzCl3v8aNGxMUFFRLUVZNUSu15WujLLmidHqKAn/8oQ4Evn07/PCDOspNacHBEBtbvMTEIMMuCVH/aJYoCwoK2L17N1OmTLEqj4+PZ+fOnRXue/PNN5OXl0dUVBQvv/xymdWxZvn5+eTn51teZ2VlAWA0GjHWYGQS877mR1OrlgA0yz2GsaDA6e5HlY7X2dk13sJCdPv2oduxA90PP6DbuRNdRobVJopOB506YerZE+X6Qtu2tt9jOfG42vkF14tZ4nWs+hhvVffVKUpZnbkcLzU1lWbNmrFjxw569eplKX/99df5+OOPOXLkiM0+R44cYdu2bXTv3p38/HyWLVvGBx98wJYtW+jbt2+Z7zNjxgxmzpxpU/7pp5/i4+Njt89z7JAfT780GHeK2LhoEXnBwXY7trgxusJCgv78k5ADBwjZv5/g33/HvVQ1apHBwKV27ciMiiIzKopL7dpR6OurUcRCCC3k5uby4IMPcuXKFavbeaVpnih37txJbGyspfxvf/sby5Yt4/fff6/ScYYNG4ZOp2PNmjVlri/rijI8PJwLFy5UeGIqYzQaSU5OJi4uDoPBwJEj4N25A605TuG336KUk7i1UjpeZ3dD8RYWotuzB93WreqyYwe6nByrTZSgIJRevVB690a57TaUbt3s2onf1c4vuF7MEq9j1cd4s7KyCAkJqTRRalb1GhISgpubG+np6VblGRkZhIaGVvk4PXv25JNPPil3vaenJ55l/IdoMBjs8mMwH6dRI9hDW1pzHP3xk+gHDarxsR3BXp+7tpQZr6KoU0mtXw/Jyep9xuxs622Cg9U5Ffv3h3790HXujK4Wxj91tfMLrhezxOtY9Snequ6nWaL08PCge/fuJCcnM2LECEt5cnIyw4cPr/Jx9uzZQxMnGM0kKAiO0QbYSMHBo3hVtoO4MXl5sHUrrFunJsg//7ReHxRUnBgHDFCHfpOBwYUQdqBpq9fJkyczatQoYmJiiI2NZeHChaSkpDB+/HgApk6dytmzZ1m6dCmgtopt2bIlnTp1oqCggE8++YRVq1axatUqLT8GoNbinTK0ASMUHZGWr/bgff48+oULYcMG2LwZrl0rXmkwqIlxyBAYOBC6dJFuGUIIh9A0UY4cOZLMzExmzZpFWloa0dHRrF+/noiICADS0tJIKdHJu6CggOeee46zZ8/i7e1Np06dWLduHYmJiVp9BCvpfm3hEuj/lL6U1VJUpI6NunYt7mvWEH/woPX6Zs0gMRHuuENNjv7+2sQp6pyioiK7t/Y0Go24u7uTl5dHUVGRXY/tCHUxXoPBgJsd/oDWfGSepKQkkpKSyly3ZMkSq9cvvPACL7zwQi1EVT2ZDdrAJfA4fUy9j+ZkXUScUlaWep9x7Vq1SvX8eUCdf0XR61FiY9EPHaomyM6d5ZwKu1IUhfT0dC47YCJZRVEICwvj9OnT6Fzgd1tX4w0KCiIsLKxGn0nzRFmX5DSKpOi4Hre8XEhLU2eHELZOnIBvvlGT45Yt1n0TAwMhIYHChAQ26fXEjRyJ3oUaFgjXYk6SjRs3xsfHx64JwmQykZOTg5+fH3oXuF9e1+JVFIXc3FwyrveZrklbFkmUduQX7MEpImjFCTh2TBKlWVGROlbq2rVqgixdpdq2LQwbpi69e4PBgGI0Yly/Xpt4Rb1QVFRkSZINGza0+/FNJhMFBQV4eXm5TOKpa/F6e3sDam+Kxo0bV7saVhKlHQUFwVHaqony6FFwsr6UterKFdi4UU2M69dbZtkA1EY3t91WnBzbtdMuTlFvme9J2nPgEeF8zN+v0WiUROkMGjRQu4gMZpN6RVnfnDsHq1fDqlVqV46SkxU3aAAJCWpiHDxYfS2EE3CF+3Gi+uzx/UqitCPzFSWgXlHWB2lpanJcuRK2bVMbMZl16KAmxqFDoVcvcJefmxDC9cj/XHYUFAT7UKfbqtNXlKmp6lXjypXqzBslk+Mtt8B998Fdd0GbNpqFKISouv79+9O1a9cyx8UWkijtyuqK8lgd6yKSk6MmxsWLbZNjz55qcrznHrjeB1YIYX+VVSOOHj3apltdVaxevRo3NzdqMvT3mDFj+Pjjj23KBw8ezIYNG6p9XGcgidKOgoLgBJEUocft6lVITwcnGF6v2hRFncT4o49gxQq4erV4XWxscXKsYJJtIYT9pKWlWZ6vWLGCV1991WqmJXMrTzOj0Vil8UyDg4MxmUyWaQira8iQISxevNiqrKyxtiuKr6oxV+VY9uL8bYBdSFAQGPEgzXD9qspVq1/T0+HNN6FjR7W7xqJFapJs2xbmzIGUFNi5E555RpKkELUoLCzMsgQGBqLT6Syv8/LyCAoK4vPPP6d///54eXnxySefkJmZyQMPPEDz5s3x8fGhc+fOfPbZZ1bH7d+/P88884zldcuWLXn99dcZO3Ys/v7+tGjRgoULF1Yan6enp1WMYWFhNCjRcE+n0/HBBx8wfPhwfH19mT17NjNmzOCmm25i0aJFtGrVCk9PTxRFISUlheHDh+Pn50dAQAD3338/586dsxxr5syZ9OnTx2Y/R7jhRFlYWIi7uzsHDhxwRDwuzfx7OK6/fm/OlRr0FBWp/RyHD4fmzeHFF+HIEfDxgTFj1IY6R47AlCkQHq51tEI4hKKofxPW9mLP/99ffPFF/vrXv3L48GEGDx5MXl4e3bt355tvvuHAgQM8/vjjjBo1ip9++qnC4/zjH/8gJiaGPXv2kJSUxJNPPlnl6Q8rMn36dIYPH87+/fsZO3YsAMeOHePzzz9n1apV7N27F4C77rqLixcvsnXrVpKTk/nzzz8ZOXKk1bFOnDjBypUrrfZzhBuuenV3dyciIsIlxgKsbUFB6uORojb0Jdk1rijT0uDf/4Z//QtOny4uj42FsWPh/vuhBvN2CuFKcnPBz89eR9MDQVXaMicH7DVv+KRJk7j77rutyp577jnL84kTJ7JhwwZWrlzJrbfeWu5xEhMTLcOLvvjii7zzzjts2bKFDh06lLvPN998g1+pE/jiiy/yyiuvWF4/+OCDlgRpVlBQwLJly2jUqBGgziL122+/ceLECcKv/2G+bNkyOnXqxK5du+jRo4dlv6VLl97Q1IzVUa17lC+//DJTp07lk08+ITg42N4xuSxzojxU6ORdRBQFvvsOPvgAvvqquL9jw4bq1ePYsRAVpWWEQohqiomJsXpdVFTEG2+8wYoVKzh79qxlMnvfSjJzly5dLM/NVbzm4eDKM2DAABYsWGBVVjpHlI4PICIiwpIkAQ4fPkx4eLglSQJERUURFBTE4cOHLYkyPDzcaj9HqVaifO+99zh27BhNmzYlIiLC5oT/+uuvdgnO1QQGqo/HnLSLiCE7G/2776pXj3/8UbyiVy948km4917wkpk0Rf3l46Ne3dmDuXFMQEBApUPC2XNwoNL/H//jH//gnXfeYd68eXTu3BlfX18mTZpEQUFBhccp3TBGp9NhMpkqfe82lXQLKytBly5TFKXMFr6ly2trVKVqJcq77rrLzmHUDQaDWn1y9GqJK0pn6CKyaxdu//wng5cvx838j8PPD0aNgvHj1bkchRDodParAjWZ1Fv/vr7aziG+fft2hg8fzsMPP3w9LhNHjx6lY8eO2gVViaioKFJSUjh9+rTlqvLQoUNcuXJFk7irlSinT59u7zjqjKAgOHE1EkWvR3f1qjqsW1hY7Qdy7RosXw7z58Mvv1habSlduqBLSoIHH5T5HIWoB9q0acOqVavYuXMnDRo0YO7cuaSnpzsk4eTn55Oenm5V5u7uTkhIyA0d5/bbb6dLly489NBDzJs3j8LCQpKSkujXr1+ZVbeOVqN+lLt37+bw4cPodDqioqK4+eab7RWXywoKgrNnPclr3ALv9JPqVWVtJspjx9R7j4sWwaVLapmHB6Z77+WHLl2IfeYZDB4etRePEEJTr7zyCidOnGDw4MH4+Pjw+OOPc9ddd3HlyhW7v9eGDRtsprNq3779DbeW1el0fPXVV0ycOJG+ffui1+sZMmQI//d//2fPcKtOqYZz584pAwYMUHQ6ndKgQQMlKChI0el0ysCBA5WMjIzqHLLWXLlyRQGUK1eu1Og4BQUFyldffaUUFBRYld92m6KAoqR3vl19smhRjd6nSgoLFWXNGkUZMkR9T/MSEaEoc+YoSkZGufE6K4nX8VwtZnvHe+3aNeXQoUPKtWvX7HK80oqKipRLly4pRUVFDjm+vdXVeCv6nquaD6pVcz5x4kSysrI4ePAgFy9e5NKlSxw4cICsrCz++te/2jeTuxhzy9fMhrXQ8nXfPnjuObVf4513woYN6k2WhAS1T+Sff6r9HmuhVZgQQtRV1ap63bBhA99++61VHXdUVBTvv/8+8fHxdgvOFZkT5Tm/NkSB/Vu+nj0Ln34Ky5bB/v3F5cHBareO8eOhdWv7vqcQQtRj1UqUJpOpzDH1DAZDpc2H6zpzojzjff2KcvNmmDxZ7YLRu3f1xn7NyVGnslq2TD2eeRgPDw91GquHH4bERPW1EEIIu6pWohw4cCBPP/00n332GU2bNgXg7NmzPPPMMwwaNMiuAboac6I87BsD3t5w8SK88466AERGFifNXr0gOlotT01Vx1A9dcr28c8/IS+v+E1uu03t2nHffTIBshBCOFi1EuU///lPhg8fTsuWLQkPD0en05GSkkLnzp355JNP7B2jSzEnyhRjEzXBffedOoD4jh1qVemJE+ryn/+oG/r4QH6+2uGqIm3bqsnxoYegVSuHfgYhhBDFqpUow8PD+fXXX0lOTub3339HURSioqK4/fbb7R2fyzFf4F2+jFrN+tBD6gKQlQU//aQmzZ074ccfITtbXefurjbKadFCndOx5GPLlmqi1HrgAiGEqIduOFEWFhbi5eXF3r17iYuLIy4urkYBzJ8/n7feeou0tDQ6derEvHnz6NOnT6X77dixg379+hEdHe3QUeNvlPmK0tyF0UpAAMTFqQuoV5FHj6od/8PCwM2ttsIUQghRRTfcPcSes4esWLGCSZMmMW3aNPbs2UOfPn1ISEggJSWlwv2uXLnCI4884pT3Q82J8vLlKmzs5gYdOkCzZpIkhRDCSVWrH6V59pCLFy/W6M3nzp3LuHHjeOyxx+jYsSPz5s0jPDzcZvT50p544gkefPBBYmNja/T+jnBDiVIIIYTT02z2kIKCAnbv3s2UKVOsyuPj49m5c2e5+y1evJg///yTTz75hNmzZ1f6PuYpZcyysrIAMBqNGI3GSvcvj3nf0sdQT4WBy5cVjMbCah/f3sqL11lJvI7najHbO16j0YiiKJhMJod0a1Oud+Myv4ezq6vxmkwmFEXBaDTiVqrmrqq/Jc1mD7lw4QJFRUU2E26GhobaDKprdvToUaZMmcL27dtxd69a6HPmzGHmzJk25Zs2bbLLFC3JyclWr7OzDUAiubk6vv76vxgMdpy63A5Kx+vsJF7Hc7WY7RWvu7s7YWFh5OTkVDrlVE1kmxvs2UGDSrqDPfDAA8yfP79ax+7SpQtPPvkkTz75ZKXbnS45yft1r776Ks8880y13rsmKju/BQUFXLt2jW3btlFYaH3xkpubW6X3qFZjHoCxY8daTapZXaXnHFPKmYesqKiIBx98kJkzZ9KuXbsqH3/q1KlMnjzZ8jorK4vw8HDi4+MJCAiodtxGo5Hk5GTi4uKsBl8oKlJ7cQD06pXgNKPHlRevs5J4Hc/VYrZ3vHl5eZw+fRo/Pz+8HDAPq6IoZGdn4+/vX+b/adVx9uxZy/PPP/+c6dOnc/jwYUuZt7d3tf9fM8+ZWVm8er2emTNn8thjj1mV+/v7lznXpKIoFBUV2VzcFBQU4FGNQVLM+1X1/Obl5eHt7U3fvn1tvmdzDWNlbjhRuru78/bbbzN69Ogb3dVKSEgIbm5uNlePGRkZNleZoP7V8Msvv7Bnzx4mTJgAFF9Su7u7s2nTJgYOHGizn6enJ56enjblBoPBLv/YSh/HYFAbt2ZlQU6OgevjMTgNe33u2iLxOp6rxWyveIuKitDpdOj1+konVq4Oc3Wg+T3soWmJ/1CCgoLQ6XRWZWvXrmXGjBkcPHiQpk2bMnr0aKZNm2ZJUjNmzGDRokWcO3eOhg0bcu+99/Lee+/Rv39/Tp06xUsvvcRLL70EFFdtliUgIMDqfUvasmULAwYMYMOGDUybNo3ffvuNjRs3MnPmTKKjo/Hw8GDp0qV06tSJrVu3snXrVp5//nn27dtHcHAwo0ePZvbs2ZaY+/fvX+Z+VT2/er0enU5X5u+mqr+jalW9Dho0iC1btjBmzJjq7A6Ah4cH3bt3Jzk5mREjRljKk5OTGT58uM32AQEB7C85tilq15LvvvuOL774gsjIyGrHYm9BQWqilAY9QrgYRYEqVsdVymSCq1fVFu2VJUofnxr3k964cSMPP/ww7733Hn369OHPP//k8ccfB9Q5hL/44gveeecdli9fTqdOnUhPT2ffvn0ArF69mq5du/LII4/w1FNP2SWxv/DCC7z99tu0atWKoOutHD/++GOefPJJduzYgaIonD17lsTERMaMGcPSpUv5/fff+ctf/oKXlxczZsywHKv0frWtWokyISGBqVOncuDAAbp3725zuX3nnXdW6TiTJ09m1KhRxMTEEBsby8KFC0lJSWH8+PGAWm169uxZli5dil6vJ9o83Nt1jRs3xsvLy6Zca0FB6uhzkiiFcDG5ueDnZ5dD6YGgqm6ck2NuCVhtf/vb35gyZYqltq9Vq1a89tprvPDCC0yfPp2UlBTCwsK4/fbbMRgMtGjRgltuuQWA4OBg3Nzc8PPzIywsrNJE+eKLL/Lyyy9blX3zzTf079/f8nrWrFk2/ezbtGnDm2++aXk9bdo0wsPD+ec//4lOp6NDhw6kpqby4osv8uqrr1riKL1fbatWojTf7J07d67NOp1OV+U+liNHjiQzM5NZs2aRlpZGdHQ069evJyIiAoC0tLRK+1Q6I+kiIoSobbt372bXrl387W9/s5QVFRWRl5dHbm4u9913H/PmzaNVq1YMGTKExMREhg0bVuWGkSU9//zzNjWKzZo1s3odExNjs1/pssOHDxMbG2t1j7F3797k5ORw5swZWrRoUe6xalO1Zw+xl6SkJJKSkspct2TJkgr3nTFjhtXlubOQRCmEi/LxUa/u7MBkMpGVlUVAQEDlVZl2aIFvMpmYOXMmd999t806Ly8vwsPDOXLkCMnJyXz77bckJSXx1ltvsXXr1hu+5xsSEkKbNm0q3Kashj2ly8pqvGmuWi1ZXtaxatMNJcrExEQ+++wzAgMDAfVS/6mnnrLUP2dmZtKnTx8OHTpk90BdidV4r0II16HT1bgK1MJkUpvB+/pWfo/SDrp168aRI0cqTGDe3t7ceeed3HnnnTz11FN06NCB/fv3061bNzw8POwy4tqNiIqKYtWqVVYJc+fOnfj7+9tcoWrphhLlxo0brTrv//3vf+eBBx6wJMrCwkKOHDli1wBdUYXjvQohhAO8+uqrDB06lPDwcO677z70ej2//fYb+/fvZ/bs2SxZsoSioiJuvfVWfHx8WLZsGd7e3pZbXREREezcuZOzZ8/i7e1NSEhIue+VnZ1t02PBx8fnhrumJCUlMW/ePCZOnMiECRM4cuQI06dPZ/LkyQ5piVxdNxRJ6dZGWrQ+cgVS9SqEqG2DBw/mm2++ITk5mR49etCzZ0/mzp1rSYRBQUH861//onfv3nTp0oXNmzezdu1aGjZsCMDMmTNJSUmhbdu2NKqkA/irr75KkyZNrJYXXnjhhmNu1qwZ69ev5+eff6Zr166MHz+ecePG2TQU0lq17lGKikmiFEI42pgxY2wa1AwePJjBgweXuf1dd91V4ahqPXv25Icffqj0nurJkycrjKt///5lXkRt2bKlzO379evHzz//XO7xytuvNt3QFaVOp7O58WqvESfqEkmUQghRd9zQFaWiKIwZM8Yy0k1eXh7jx4+3tEgqef+yPpNEKYQQdccNJcrSw9Y9/PDDNts88sgjNYuoDpBWr0IIUXfcUKJcvHixo+KoU+SKUggh6g7naX9bh0j3ECFch7Ter9vs8f1KonQAc6LMz4e8PE1DEUKUwzwaTVXnJBSuyfz91mTGGeke4gD+/uoAH4qiVr+GhWkdkRCiNDc3N4KCgsjIyADUDvP2bMVvMpkoKCggLy/PqTrPl6euxasoCrm5uWRkZBAUFISbm1u130sSpQPo9RAYqCZJSZRCOK+w6/84zcnSnhRF4dq1a3h7e7tEN7q6Gm9QUJDle64uSZQO0qBBcaIUQjgnnU5HkyZNaNy4MUaj0a7HNhqNbNu2jb59+7rExNh1MV6DwVCjK0kzSZQOIi1fhXAdbm5udvkPtfQxCwsL8fLyconEI/GWz/krol2UtHwVQoi6QRKlg8gVpRBC1A2SKB1EEqUQQtQNkigdRBKlEELUDZIoHUQSpRBC1A2SKB1EBkYXQoi6QRKlg0irVyGEqBskUTqIVL0KIUTdIInSQcyJMjNT0zCEEELUkOaJcv78+URGRuLl5UX37t3Zvn17udv+8MMP9O7dm4YNG+Lt7U2HDh145513ajHaqmvbFtzd4fhx2LFD62iEEEJUl6aJcsWKFUyaNIlp06axZ88e+vTpQ0JCAikpKWVu7+vry4QJE9i2bRuHDx/m5Zdf5uWXX2bhwoW1HHnlwsJg7Fj1+fTp2sYihBCi+jRNlHPnzmXcuHE89thjdOzYkXnz5hEeHs6CBQvK3P7mm2/mgQceoFOnTrRs2ZKHH36YwYMHV3gVqqVp08BggM2bYetWraMRQghRHZoNil5QUMDu3buZMmWKVXl8fDw7d+6s0jH27NnDzp07mT17drnb5Ofnk5+fb3mdlZUFqCPP12S2APO+FR2jSRMYO1bPhx+68corJr79tgitZq+pSrzOROJ1PFeLWeJ1rPoYb1X31SmKolT7XWogNTWVZs2asWPHDnr16mUpf/311/n44485cuRIufs2b96c8+fPU1hYyIwZM3jllVfK3XbGjBnMnDnTpvzTTz/Fx8enZh+iCi5c8GL8+NspLHRj1qwddOlyweHvKYQQonK5ubk8+OCDXLlyhYCAgHK303yardITbiqKUumkodu3bycnJ4cff/yRKVOm0KZNGx544IEyt506dSqTJ0+2vM7KyiI8PJz4+PgKT0xljEYjycnJxMXFVTrFy5498P77sHFjLC++qM1V5Y3E6wwkXsdztZglXseqj/Gaaxgro1miDAkJwc3NjfT0dKvyjIwMQkNDK9w3MjISgM6dO3Pu3DlmzJhRbqL09PTE09PTptxgMNjlx1CV47z0Enz0EezYoWfrVj1xcTV+22qz1+euLRKv47lazBKvY9WneKu6n2aNeTw8POjevTvJyclW5cnJyVZVsZVRFMXqHqQzatoUxo9Xn7/6KmhT2S2EEKI6NK16nTx5MqNGjSImJobY2FgWLlxISkoK469nlalTp3L27FmWLl0KwPvvv0+LFi3o0KEDoParfPvtt5k4caJmn6GqXnwRPvwQfvwRNmyAhAStIxJCCFEVmibKkSNHkpmZyaxZs0hLSyM6Opr169cTEREBQFpamlWfSpPJxNSpUzlx4gTu7u60bt2aN954gyeeeEKrj1BlYWHw1FPw9tvqVeWQIWjWAlYIIUTVad6YJykpiaSkpDLXLVmyxOr1xIkTXeLqsTwvvAALFsAvv8A338CwYVpHJIQQojKaD2FXnzRqBBMmqM+nT5d7lUII4QokUday554DPz+1y8jXX2sdjRBCiMpIoqxlISHw9NPq8+nTwWTSNh4hhBAVk0SpgcmTISAAfvsNVq/WOhohhBAVkUSpgeBgeOYZ9fmMGXJVKYQQzkwSpUYmTYLAQDh4EFas0DoaIYQQ5ZFEqZGgIHj2WfV5UhLs369pOEIIIcohiVJDzz4LsbFw+TIMHgwnTmgdkRBCiNIkUWrIx0cdeCA6GtLSIC4Ozp3TOiohhBAlSaLUWHAwbNwIkZHw55/qleXly1pHJYQQwkwSpRNo2hQ2bYLQUNi3D+68E65d0zoqIYQQIInSabRpo15ZBgbC9u0wciQYjVpHJYQQQhKlE+naFdauBS8v9XHcOOljKYQQWpNE6WT69IGVK8HNDZYtU1vGyuDpQgihHUmUTmjoUFi8WH0+bx7MmaNpOEIIUa9JonRSo0apSRJg2jR1Lktp4COEELVPEqUTe/ppeOUV9flbb8FNN8EPP2gakhBC1DuSKJ3crFnw1VfQpAn88Qf07QsTJ0J2ttaRCSFE/SCJ0gUMHw6HDsHYsWrDnn/+Ux3NZ9MmrSMTQoi6TxKliwgKgo8+UpNjy5aQkqKO4vPoo3DpktbRCSFE3SWJ0sXExakzjUycCDodLFkCUVHw5ZfSjUQIIRxBEqUL8vOD995TR/Bp3x7S0+Huu+GWW+Dzz6GwUOsIhRCi7pBE6cJ694a9e2HqVHU0n19+UYe+a98e3n8fcnO1jlAIIVyf5oly/vz5REZG4uXlRffu3dm+fXu5265evZq4uDgaNWpEQEAAsbGxbNy4sRajdT5eXvD663DqFLz6KjRsCMePw4QJ0KIFTJ8O589rHaUQQrguTRPlihUrmDRpEtOmTWPPnj306dOHhIQEUlJSytx+27ZtxMXFsX79enbv3s2AAQMYNmwYe/bsqeXInU/jxjBzptrI55//hFatIDNT7V7SogVMnKgnNdVX6zCFEMLlaJoo586dy7hx43jsscfo2LEj8+bNIzw8nAULFpS5/bx583jhhRfo0aMHbdu25fXXX6dt27asXbu2liN3Xj4+8NRTcOQIrFgBMTGQlwcffuhGUtLtDBjgxocfwsWLWkcqhBCuwV2rNy4oKGD37t1MmTLFqjw+Pp6dO3dW6Rgmk4ns7GyCg4PL3SY/P5/8/HzL66ysLACMRiPGGsxjZd63JsdwtBEj4K67YNs2HW+/rWPTJj07dujZsQMmTlRISFB48EETiYkKXl5aR2vNFc5vSa4WL7hezBKvY9XHeKu6r05RtOlUkJqaSrNmzdixYwe9evWylL/++ut8/PHHHDlypNJjvPXWW7zxxhscPnyYxo0bl7nNjBkzmDlzpk35p59+io+PT/U/gAvKzPRi+/ZmbNkSzsmTgZZyX98CevVKpX//M3TsmIle8zvXQgjheLm5uTz44INcuXKFgICAcrfT7IrSTKfTWb1WFMWmrCyfffYZM2bM4Ouvvy43SQJMnTqVyZMnW15nZWURHh5OfHx8hSemMkajkeTkZOLi4jAYDNU+Tm0xx/v++60wGAzs32/ks8/0LF+u58wZD5KTW5Kc3JJmzRSGDTNx550KffsqeHhoG6+rnV9XiRdcL2aJ17HqY7zmGsbKaJYoQ0JCcHNzIz093ao8IyOD0NDQCvddsWIF48aNY+XKldx+++0Vbuvp6Ymnp6dNucFgsMuPwV7HqS3meLt1g27d4O9/h61b4ZNP4Isv4OxZHR984MYHH0BgICQmqtW3Q4ZADf6uqHG8rsLV4gXXi1nidaz6FG9V99Osks3Dw4Pu3buTnJxsVZ6cnGxVFVvaZ599xpgxY/j000+54447HB1mnafXw4AB6vB4587BN9/AX/6itqK9cgU++0ztm9moESQkwAcfwJkzWkcthBC1R9O7UZMnT+bf//43ixYt4vDhwzzzzDOkpKQwfvx4QK02feSRRyzbf/bZZzzyyCP84x//oGfPnqSnp5Oens6VK1e0+gh1ipcX3HEHLFwIqamwY4c6D2bbtlBQABs2wJNPQng4dOoEkyfDxo0yT6YQom7T9B7lyJEjyczMZNasWaSlpREdHc369euJiIgAIC0tzapP5YcffkhhYSFPPfUUTz31lKV89OjRLFmypLbDr9Pc3KBXL3V54w34/Xf4+mt1+ekndTaTQ4fgnXfUBNu3rzpI++DB6tizVbjNLIQQLkHzxjxJSUkkJSWVua508tuyZYvjAxI2dDro2FFdpkxR+2B++616NblxI5w9q85qsmkTPPssNGsGt98OgwapS9OmWn8CIYSoPs0TpXA9wcFw//3qoihw+HBx0ty6VU2cH3+sLgAdOsDAgWrSHDAAGjTQNn4hhLgRkihFjeh0alVrVBQ884x6v/KHH2DzZnXZvVuttv39d5g/X92+Wzc1afbtqw7sHhSk9acQQojySaIUduXtrc6ZGRenvr50CbZsKU6cv/+uJs/du+HNN9XEGR0NffrAbbepj82ba/oRhBDCiiRK4VANGqhD6Y0Yob4+exa+/15dtm+Ho0fViaj371evOAEiIqB3bzf8/VsSEqLj5pvVMWyFEEILkihFrWrWDB5+WF1A7bv5ww/qsn077NmjThl26pQe6MqHH6p9Pdu1g5tugq5d1cebboKwMO0+hxCi/pBEKTQVGgr33KMuANnZ8OOPsHVrEevWZZKa2oiMDJ3lPufy5db7dumi9uk03yeNipLGQkII+5JEKZyKv796f7N/fxM9evyPxMREMjMN7N2LZdm3T51G7Nw5SE5Wl5LCwmyTZ8eO6uhCQghxoyRRCqcXFqaONTtkSHHZ1avqfc0DB4oHPzh0CE6fhvR0ddm82fo4DRsWJ82OHYufN28uAyQIIconiVK4JF9f6NlTXUrKylKraM2J8+BBtZ/nyZOQmaneB92+3XofPz+1r6d56dhRfWzTBs1mTxFCOA9JlKJOCQiAW25Rl5Jyc9Xq2sOH1eXQIfXx6FHIyYFfflGXktzcoFWr4gTavr26tGunVuPKVagQ9YMkSlEv+PjAzTerS0lGIxw7Vjwognk5fFhtWHT0qLqsXWu9X1BQcdJs3x5at9Zx7lwAubnq9GRCiLpDEqWo1wyG4nuWJSkKpKVZJ84//lCvSlNS4PJldXD4n34y7+EODGDSJLULTNu2tkvr1uoA8kII1yKJUogy6HTqYO5Nm6rj1JZ07Zp6FXrkiLr88Qf8/ruJgwcLuXrVg7Nn1YEVSo/hr9OpDYciI9WlZUt1MT9v1gzc5V+kEE5H/lkKcYO8vaFzZ3UxMxqLWLfuv/TsmcjJkwZLlW3JJStLbZV7+jRs22Z7XHd3da7PiAg1oYaHq4v5efPmastduTcqRO2SRCmEneh0aiILC7NtjasocOGCeiV68qS6nDhR/HjqlHq/9MQJdSmPl5eaMJs3V69AzUvJ12FhcmUqhD3JPychaoFOp7aUbdQIYmNt15tMkJqqJknzVeeZM9bPMzIgL09NtseOlf9eer06alHJRFrW4u3tuM8rRF0iiVIIJ6DXF18plicvT02mp09juQ9qXs6cUR/T0qCwUH1MS7Pt8lKSn587AQGDiIx0o0kT9UrUvISGFj9v1Ag8Pe3/mYVwFZIohXARXl5qv85WrcrfxmRSrzxLJ9LSy5UrkJOjIyfHj9TUyt87IEBNmI0bWz+al4YN1Qm9zUtQkNoPVYi6QBKlEHWIXl98Jdi9e/nbXb0KJ08a+frrn4iM7MmFC+6Wof9KLufOQVGR2hApKwv+/LPqsQQFqUmzYUN1oPrgYPWxoudBQepISdJgSTgTSZRC1EO+vupgCZ06ZZKYqGAwlL2dyaT2GT1/Xr1SPX/e9vn583DxYvGSna3ue/myuhw/fmOx6fVqwiy9BAS4cfFiJ3bv1luuWgMD1aXk88BAyv08QlSHJEohRLn0+uLq1Pbtq7aP0QiXLlknz8xMtcxcXvKx5PPCQjU5m/crFQ3QhjVrKo/B21utLg4MLH4s+TwgoPwlMFCdxcbfXxKuUEmiFELYlcGg3sNs3PjG9lMUdTAH85XolSvFzy9fhszMIvbsOU7Dhq3IznazrL9ypXjJyVGPde2aupw7V7PP4uWlJsyAgOLkWfK5n1/xo3kxv/by0nHqlD8nTqjVymqZVCu7IkmUQginoNOpY/L6+KgjIpVmNJpYv/4QiYktMRjKbilUWKhW/ZoTZ1aW9aN5yc4uvu9a1nLtmnq8vDx1OX++Op/IHbAe1kmnUxOmr2/xo6+v+pnNj6Wfl1y8vct/9PJSWyd7eal/rEhCth/NE+X8+fN56623SEtLo1OnTsybN48+ffqUuW1aWhrPPvssu3fv5ujRo/z1r39l3rx5tRuwEMJpubsXNwyqCaNRTabmJSur7Oc5Oepifm5dpnDpUgEFBR5cu6ZmLUUp3tfRvLysk6enpzptXOnFYFAf3d3dOH++G6tWueHpqZ5Lg6H4sfTzkkt55dVZPDycr8W0polyxYoVTJo0ifnz59O7d28+/PBDEhISOHToEC1atLDZPj8/n0aNGjFt2jTeeecdDSIWQtQHBkPxvdnqMhoLWb9+A4mJiej1BnJz1dbGOTnqo/m5ubzkY+mya9fU5xU9FhRYv7/5arjq9EB49T+wHel0lSdid3d37rmnEYmJjo9H00Q5d+5cxo0bx2OPPQbAvHnz2LhxIwsWLGDOnDk227ds2ZJ3330XgEWLFtVqrEIIUV1ubsX3NR3FZFKTZV4e5OdbP5qfG43qNmUt164V8dtvh2jTJgpFccNoVLcvLMTyvKzXlZVXtpRO8KBeeZvjKp+OxMTaSWGaJcqCggJ2797NlClTrMrj4+PZuXOn3d4nPz+f/Px8y+usrCwAjEYjRqOx2sc171uTY9QmidexXC1ecL2YJd7KubkV3/e8UUajkeTk48TFtcZQi819FUXtq1s6eZaVhAsLdZaya9cKuXjxol3+H6+MZonywoULFBUVERoaalUeGhpKenq63d5nzpw5zJw506Z806ZN+Pj41Pj4ycnJNT5GbZJ4HcvV4gXXi1nidSxXijcoqGbx5ubmVmk7zRvz6Eo1zVIUxaasJqZOncrkyZMtr7OysggPDyc+Pp6AgIBqH1f96yuZuLi4Wv3rq7okXsdytXjB9WKWeB2rPsZrrmGsjGaJMiQkBDc3N5urx4yMDJurzJrw9PTEs4wRnQ0Gg11+DPY6Tm2ReB3L1eIF14tZ4nWs+hRvVffTV+voduDh4UH37t1tLpuTk5Pp1auXRlEJIYQQ1jStep08eTKjRo0iJiaG2NhYFi5cSEpKCuPHjwfUatOzZ8+ydOlSyz579+4FICcnh/Pnz7N37148PDyIiorS4iMIIYSo4zRNlCNHjiQzM5NZs2aRlpZGdHQ069evJyIiAlAHGEhJSbHa5+abb7Y83717N59++ikRERGcPHmyNkMXQghRT2jemCcpKYmkpKQy1y1ZssSmTFEUB0ckhBBCFNPsHqUQQgjhCjS/oqxt5ivSqjYLLo/RaCQ3N5esrCyXaCEm8TqWq8ULrhezxOtY9TFecx6orKay3iXK7OujEYeHO8eYhkIIIbSVnZ1NYGBguet1Sj276WcymUhNTcXf379GAxuYBy44ffp0jQYuqC0Sr2O5WrzgejFLvI5VH+NVFIXs7GyaNm2KXl/+nch6d0Wp1+tp3ry53Y4XEBDgEj8qM4nXsVwtXnC9mCVex6pv8VZ0JWkmjXmEEEKICkiiFEIIISogibKaPD09mT59epnjyDojidexXC1ecL2YJV7HknjLV+8a8wghhBA3Qq4ohRBCiApIohRCCCEqIIlSCCGEqIAkSiGEEKICkiirYf78+URGRuLl5UX37t3Zvn271iEBMGPGDHQ6ndUSFhZmWa8oCjNmzKBp06Z4e3vTv39/Dh48WKsxbtu2jWHDhtG0aVN0Oh1fffWV1fqqxJifn8/EiRMJCQnB19eXO++8kzNnzmgS75gxY2zOec+ePTWJd86cOfTo0QN/f38aN27MXXfdxZEjR6y2cabzW5V4nen8AixYsIAuXbpYOrnHxsby3//+17Lemc5vVeJ1tvNb2pw5c9DpdEyaNMlSpsk5VsQNWb58uWIwGJR//etfyqFDh5Snn35a8fX1VU6dOqV1aMr06dOVTp06KWlpaZYlIyPDsv6NN95Q/P39lVWrVin79+9XRo4cqTRp0kTJysqqtRjXr1+vTJs2TVm1apUCKF9++aXV+qrEOH78eKVZs2ZKcnKy8uuvvyoDBgxQunbtqhQWFtZ6vKNHj1aGDBlidc4zMzOttqmteAcPHqwsXrxYOXDggLJ3717ljjvuUFq0aKHk5ORYtnGm81uVeJ3p/CqKoqxZs0ZZt26dcuTIEeXIkSPKSy+9pBgMBuXAgQOKojjX+a1KvM52fkv6+eeflZYtWypdunRRnn76aUu5FudYEuUNuuWWW5Tx48dblXXo0EGZMmWKRhEVmz59utK1a9cy15lMJiUsLEx54403LGV5eXlKYGCg8sEHH9RShNZKJ56qxHj58mXFYDAoy5cvt2xz9uxZRa/XKxs2bKjVeBVF/Y9m+PDh5e6jZbwZGRkKoGzdulVRFOc/v6XjVRTnPr9mDRo0UP797387/fktHa+iOO/5zc7OVtq2baskJycr/fr1syRKrc6xVL3egIKCAnbv3k18fLxVeXx8PDt37tQoKmtHjx6ladOmREZG8v/+3//j+PHjAJw4cYL09HSr2D09PenXr5/TxF6VGHfv3o3RaLTapmnTpkRHR2v2ObZs2ULjxo1p164df/nLX8jIyLCs0zLeK1euABAcHAw4//ktHa+Zs57foqIili9fztWrV4mNjXX681s6XjNnPL9PPfUUd9xxB7fffrtVuVbnuN4Nil4TFy5coKioiNDQUKvy0NBQ0tPTNYqq2K233srSpUtp164d586dY/bs2fTq1YuDBw9a4isr9lOnTmkRro2qxJieno6HhwcNGjSw2UaL7yAhIYH77ruPiIgITpw4wSuvvMLAgQPZvXs3np6emsWrKAqTJ0/mtttuIzo6GnDu81tWvOCc53f//v3ExsaSl5eHn58fX375JVFRUZb/hJ3t/JYXLzjn+V2+fDm//voru3btslmn1W9YEmU1lJ6eS1GUGk3ZZS8JCQmW5507dyY2NpbWrVvz8ccfW27QO2vsJVUnRq0+x8iRIy3Po6OjiYmJISIignXr1nH33XeXu5+j450wYQK//fYbP/zwg806Zzy/5cXrjOe3ffv27N27l8uXL7Nq1SpGjx7N1q1bLeud7fyWF29UVJTTnd/Tp0/z9NNPs2nTJry8vMrdrrbPsVS93oCQkBDc3Nxs/irJyMiw+QvHGfj6+tK5c2eOHj1qaf3qzLFXJcawsDAKCgq4dOlSudtoqUmTJkRERHD06FFAm3gnTpzImjVr+P77762mlHPW81tevGVxhvPr4eFBmzZtiImJYc6cOXTt2pV3333Xac9vefGWRevzu3v3bjIyMujevTvu7u64u7uzdetW3nvvPdzd3S3vWdvnWBLlDfDw8KB79+4kJydblScnJ9OrVy+Noipffn4+hw8fpkmTJkRGRhIWFmYVe0FBAVu3bnWa2KsSY/fu3TEYDFbbpKWlceDAAaf4HJmZmZw+fZomTZoAtRuvoihMmDCB1atX89133xEZGWm13tnOb2XxlkXL81seRVHIz893uvNbWbxl0fr8Dho0iP3797N3717LEhMTw0MPPcTevXtp1aqVNue4Wk2A6jFz95CPPvpIOXTokDJp0iTF19dXOXnypNahKc8++6yyZcsW5fjx48qPP/6oDB06VPH397fE9sYbbyiBgYHK6tWrlf379ysPPPBArXcPyc7OVvbs2aPs2bNHAZS5c+cqe/bssXSvqUqM48ePV5o3b658++23yq+//qoMHDjQYc3VK4o3OztbefbZZ5WdO3cqJ06cUL7//nslNjZWadasmSbxPvnkk0pgYKCyZcsWq+b+ubm5lm2c6fxWFq+znV9FUZSpU6cq27ZtU06cOKH89ttvyksvvaTo9Xpl06ZNiqI41/mtLF5nPL9lKdnqVVG0OceSKKvh/fffVyIiIhQPDw+lW7duVs3ZtWTuT2QwGJSmTZsqd999t3Lw4EHLepPJpEyfPl0JCwtTPD09lb59+yr79++v1Ri///57BbBZRo8eXeUYr127pkyYMEEJDg5WvL29laFDhyopKSm1Hm9ubq4SHx+vNGrUSDEYDEqLFi2U0aNH28RSW/GWFSegLF682LKNM53fyuJ1tvOrKIoyduxYy7/9Ro0aKYMGDbIkSUVxrvNbWbzOeH7LUjpRanGOZZotIYQQogJyj1IIIYSogCRKIYQQogKSKIUQQogKSKIUQgghKiCJUgghhKiAJEohhBCiApIohRBCiApIohRCCCEqIIlSCFFlOp2Or776SuswhKhVkiiFcBFjxoxBp9PZLEOGDNE6NCHqNJmPUggXMmTIEBYvXmxV5unpqVE0QtQPckUphAvx9PQkLCzMajHP5K7T6ViwYAEJCQl4e3sTGRnJypUrrfbfv38/AwcOxNvbm4YNG/L444+Tk5Njtc2iRYvo1KkTnp6eNGnShAkTJlitv3DhAiNGjMDHx4e2bduyZs0ax35oITQmiVKIOuSVV17hnnvuYd++fTz88MM88MADHD58GIDc3FyGDBlCgwYN2LVrFytXruTbb7+1SoQLFizgqaee4vHHH2f//v2sWbOGNm3aWL3HzJkzuf/++/ntt99ITEzkoYce4uLFi7X6OYWoVdWed0QIUatGjx6tuLm5Kb6+vlbLrFmzFEVRp60aP3681T633nqr8uSTTyqKoigLFy5UGjRooOTk5FjWr1u3TtHr9Up6erqiKIrStGlTZdq0aeXGACgvv/yy5XVOTo6i0+mU//73v3b7nEI4G7lHKYQLGTBgAAsWLLAqCw4OtjyPjY21WhcbG8vevXsBOHz4MF27dsXX19eyvnfv3phMJo4cOYJOpyM1NZVBgwZVGEOXLl0sz319ffH39ycjI6O6H0kIpyeJUggX4uvra1MVWhmdTgeAoiiW52Vt4+3tXaXjGQwGm31NJtMNxSSEK5F7lELUIT/++KPN6w4dOgAQFRXF3r17uXr1qmX9jh070Ov1tGvXDn9/f1q2bMnmzZtrNWYhnJ1cUQrhQvLz80lPT7cqc3d3JyQkBICVK1cSExPDbbfdxn/+8x9+/vlnPvroIwAeeughpk+fzujRo5kxYwbnz59n4sSJjBo1itDQUABmzJjB+PHjady4MQkJCWRnZ7Njxw4mTpxYux9UCCciiVIIF7JhwwaaNGliVda+fXt+//13QG2Runz5cpKSkggLC+M///kPUVFRAPj4+LBx40aefvppevTogY+PD/fccw9z5861HGv06NHk5eXxzjvv8NxzzxESEsK9995bex9QCCekUxRF0ToIIUTN6XQ6vvzyS+666y6tQxGiTpF7lEIIIUQFJFEKIYQQFZB7lELUEXIXRQjHkCtKIYQQogKSKIUQQogKSKIUQgghKiCJUgghhKiAJEohhBCiApIohRBCiApIohRCCCEqIIlSCCGEqMD/B95ihgg66weuAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 500x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# 创建图形\n",
    "plt.figure(figsize=(5, 3))\n",
    "\n",
    "plt.plot(x_axis_epoch, y_axis_train_error, label='Train Error', color='blue')\n",
    "plt.plot(x_axis_epoch, y_axis_test_error, label='Test Error', color='red')\n",
    "\n",
    "# 添加标题和标签\n",
    "plt.title('Train Error and Test Error')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Error')\n",
    "\n",
    "# 添加图例\n",
    "plt.grid(True)\n",
    "plt.legend()\n",
    "\n",
    "# 显示图形\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3093fff",
   "metadata": {},
   "source": [
    "# Dropout\n",
    "为了防止过拟合，我们可以在训练时随机丢弃部分神经元，这被称为 Dropout。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "a24a290a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys, numpy as np\n",
    "from keras.datasets import mnist\n",
    "\n",
    "# load the MNIST dataset\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "\n",
    "# 训练数据一维化，归一化\n",
    "images, labels = (x_train[0:1000].reshape(1000,28*28) / 255, y_train[0:1000])\n",
    "# 训练数据one-hot编码\n",
    "one_hot_labels = np.zeros((len(labels),10))\n",
    "for i,label in enumerate(labels):\n",
    "    one_hot_labels[i][label] = 1\n",
    "labels = one_hot_labels\n",
    "\n",
    "# 测试数据归一化\n",
    "test_images = x_test.reshape(len(x_test),28*28) / 255  \n",
    "# 测试数据one-hot编码\n",
    "test_labels = np.zeros((len(y_test),10))\n",
    "for i,label in enumerate(y_test):\n",
    "    test_labels[i][label] = 1\n",
    "\n",
    "np.random.seed(1)\n",
    "\n",
    "def relu(x):\n",
    "    return (x >= 0) * x  # ReLU激活函数\n",
    "\n",
    "def relu2deriv(output):\n",
    "    return output>=0  # ReLU函数的导数\n",
    "\n",
    "alpha, epochs, hidden_size, pixels_per_image, num_labels = (0.005, 400, 100, 784, 10)\n",
    "\n",
    "# 权重初始化\n",
    "weights_0_1 = 0.2*np.random.random((pixels_per_image,hidden_size)) - 0.1\n",
    "weights_1_2 = 0.2*np.random.random((hidden_size,num_labels)) - 0.1\n",
    "\n",
    "# 可视化数据记录\n",
    "x_axis_epoch = []\n",
    "y_axis_train_error = []\n",
    "y_axis_test_error = []\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "7f4841e2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I:0 Train-Err:0.891 Train-Acc:0.413 Test-Err:0.641 Test-Acc:0.6333\n",
      "I:10 Train-Err:0.472 Train-Acc:0.764 Test-Err:0.458 Test-Acc:0.787\n",
      "I:20 Train-Err:0.430 Train-Acc:0.809 Test-Err:0.415 Test-Acc:0.8133\n",
      "I:30 Train-Err:0.415 Train-Acc:0.811 Test-Err:0.421 Test-Acc:0.8114\n",
      "I:40 Train-Err:0.413 Train-Acc:0.827 Test-Err:0.419 Test-Acc:0.8112\n",
      "I:50 Train-Err:0.392 Train-Acc:0.836 Test-Err:0.409 Test-Acc:0.8133\n",
      "I:60 Train-Err:0.402 Train-Acc:0.836 Test-Err:0.412 Test-Acc:0.8236\n",
      "I:70 Train-Err:0.383 Train-Acc:0.857 Test-Err:0.412 Test-Acc:0.8033\n",
      "I:80 Train-Err:0.386 Train-Acc:0.854 Test-Err:0.410 Test-Acc:0.8054\n",
      "I:90 Train-Err:0.376 Train-Acc:0.868 Test-Err:0.411 Test-Acc:0.8144\n",
      "I:100 Train-Err:0.369 Train-Acc:0.864 Test-Err:0.411 Test-Acc:0.7903\n",
      "I:110 Train-Err:0.371 Train-Acc:0.868 Test-Err:0.411 Test-Acc:0.8003\n",
      "I:120 Train-Err:0.353 Train-Acc:0.857 Test-Err:0.402 Test-Acc:0.8046\n",
      "I:130 Train-Err:0.352 Train-Acc:0.867 Test-Err:0.408 Test-Acc:0.8091\n",
      "I:140 Train-Err:0.355 Train-Acc:0.885 Test-Err:0.405 Test-Acc:0.8083\n",
      "I:150 Train-Err:0.342 Train-Acc:0.883 Test-Err:0.404 Test-Acc:0.8107\n",
      "I:160 Train-Err:0.361 Train-Acc:0.876 Test-Err:0.399 Test-Acc:0.8146\n",
      "I:170 Train-Err:0.344 Train-Acc:0.889 Test-Err:0.404 Test-Acc:0.8074\n",
      "I:180 Train-Err:0.333 Train-Acc:0.892 Test-Err:0.399 Test-Acc:0.807\n",
      "I:190 Train-Err:0.335 Train-Acc:0.898 Test-Err:0.407 Test-Acc:0.8066\n",
      "I:200 Train-Err:0.347 Train-Acc:0.893 Test-Err:0.405 Test-Acc:0.8036\n",
      "I:210 Train-Err:0.336 Train-Acc:0.894 Test-Err:0.405 Test-Acc:0.8034\n",
      "I:220 Train-Err:0.325 Train-Acc:0.896 Test-Err:0.402 Test-Acc:0.8067\n",
      "I:230 Train-Err:0.321 Train-Acc:0.894 Test-Err:0.404 Test-Acc:0.8091\n",
      "I:240 Train-Err:0.332 Train-Acc:0.898 Test-Err:0.415 Test-Acc:0.8091\n",
      "I:250 Train-Err:0.320 Train-Acc:0.899 Test-Err:0.395 Test-Acc:0.8182\n",
      "I:260 Train-Err:0.321 Train-Acc:0.899 Test-Err:0.390 Test-Acc:0.8204\n",
      "I:270 Train-Err:0.312 Train-Acc:0.906 Test-Err:0.382 Test-Acc:0.8194\n",
      "I:280 Train-Err:0.317 Train-Acc:0.9 Test-Err:0.396 Test-Acc:0.8208\n",
      "I:290 Train-Err:0.301 Train-Acc:0.908 Test-Err:0.399 Test-Acc:0.8181\n",
      "I:300 Train-Err:0.309 Train-Acc:0.914 Test-Err:0.395 Test-Acc:0.8232\n",
      "I:310 Train-Err:0.305 Train-Acc:0.911 Test-Err:0.396 Test-Acc:0.8206\n",
      "I:320 Train-Err:0.289 Train-Acc:0.919 Test-Err:0.397 Test-Acc:0.8217\n",
      "I:330 Train-Err:0.314 Train-Acc:0.906 Test-Err:0.408 Test-Acc:0.8198\n",
      "I:340 Train-Err:0.314 Train-Acc:0.906 Test-Err:0.394 Test-Acc:0.824\n",
      "I:350 Train-Err:0.306 Train-Acc:0.915 Test-Err:0.412 Test-Acc:0.8195\n",
      "I:360 Train-Err:0.297 Train-Acc:0.917 Test-Err:0.394 Test-Acc:0.821\n",
      "I:370 Train-Err:0.289 Train-Acc:0.919 Test-Err:0.405 Test-Acc:0.8153\n",
      "I:380 Train-Err:0.294 Train-Acc:0.917 Test-Err:0.407 Test-Acc:0.8116\n",
      "I:390 Train-Err:0.282 Train-Acc:0.927 Test-Err:0.409 Test-Acc:0.81\n",
      "I:399 Train-Err:0.289 Train-Acc:0.909 Test-Err:0.401 Test-Acc:0.8147\n"
     ]
    }
   ],
   "source": [
    "\n",
    "for j in range(epochs):\n",
    "    error, correct_cnt = (0.0, 0)  # 训练集误差，预测正确个数计数\n",
    "    \n",
    "    for i in range(len(images)):  # SGD\n",
    "        # 前向传播\n",
    "        layer_0 = images[i:i+1]\n",
    "        layer_1 = relu(layer_0.dot(weights_0_1))\n",
    "        dropout_mask = np.random.randint(2, size=layer_1.shape) # 模拟dropout\n",
    "        layer_1 *= dropout_mask * 2                             # *2是为了保证在训练时保持神经元输出的期望不变\n",
    "        layer_2 = layer_1.dot(weights_1_2)\n",
    "\n",
    "        error += np.sum((labels[i:i+1] - layer_2) ** 2)  # 均方差\n",
    "        correct_cnt += int(np.argmax(layer_2) == np.argmax(labels[i:i+1])) # 预测正确个数计数\n",
    "\n",
    "        # 反向传播\n",
    "        layer_2_delta = (labels[i:i+1] - layer_2)\n",
    "        layer_1_delta = layer_2_delta.dot(weights_1_2.T) * relu2deriv(layer_1)\n",
    "        layer_1_delta *= dropout_mask  # 注意：在测试时，没有这一步\n",
    "        \n",
    "        # 更新权重\n",
    "        weights_1_2 += alpha * layer_1.T.dot(layer_2_delta)\n",
    "        weights_0_1 += alpha * layer_0.T.dot(layer_1_delta)\n",
    "\n",
    "    # 每隔10次迭代输出结果，并记录测试集准确率\n",
    "    if(j % 10 == 0 or j == epochs-1):\n",
    "        \n",
    "        # 测试集\n",
    "        test_error, test_correct_cnt = (0.0, 0)   # 测试集误差，预测正确个数计数\n",
    "\n",
    "        for i in range(len(test_images)):\n",
    "\n",
    "            layer_0 = test_images[i:i+1]\n",
    "            layer_1 = relu(layer_0.dot(weights_0_1))\n",
    "            layer_2 = layer_1.dot(weights_1_2)\n",
    "            \n",
    "            test_error += np.sum((test_labels[i:i+1] - layer_2) ** 2)\n",
    "            test_correct_cnt += int(np.argmax(layer_2) == \\\n",
    "                                            np.argmax(test_labels[i:i+1]))\n",
    "        \n",
    "        sys.stdout.write(\"I:\"+str(j)+ \\\n",
    "                     \" Train-Err:\" + str(error/float(len(images)))[0:5] +\\\n",
    "                     \" Train-Acc:\" + str(correct_cnt/float(len(images))))\n",
    "        sys.stdout.write(\" Test-Err:\" + str(test_error/float(len(test_images)))[0:5] +\\\n",
    "                         \" Test-Acc:\" + str(test_correct_cnt/float(len(test_images))))\n",
    "        print()  # 换行\n",
    "\n",
    "        x_axis_epoch.append(j) # 记录轮次\n",
    "        y_axis_train_error.append(error/float(len(images))) # 记录训练集误差\n",
    "        y_axis_test_error.append(test_error/float(len(test_images))) # 记录测试集误差\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74fd5aff",
   "metadata": {},
   "source": [
    "## 训练过程可视化\n",
    "与之前的可视化图对比可见测试误差被有效的控制下来。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "591965b2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcoAAAE6CAYAAACWDhLFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABc40lEQVR4nO3dd3xT1fsH8M9NmqZ7UeimLWUVypCWURBZtkJliQoOloBSK0twMFTKEBAR8KuAgGxQEEFU4AdUmYIIVqaFyi6UlkKB7jZpcn5/HBKaJk3TNmkGz/v1uq8mN/fePLlJ8+Sce4bAGGMghBBCiE4icwdACCGEWDJKlIQQQogelCgJIYQQPShREkIIIXpQoiSEEEL0oERJCCGE6EGJkhBCCNGDEiUhhBCiByVKQgghRA9KlMSkBEEwaDl48GCNnicxMRGCIBgn6Ef0xTt8+HCjPpc1quycHzx40OD33xhSUlKQmJiI69evG7T92rVrTfqZJLbDztwBENv2559/atyfNWsWDhw4gP3792usb9asWY2eZ9SoUejZs2eNjqHLSy+9hEmTJmmtr1u3rtGfy9a0adNG6/1/4YUXEBYWhgULFhj9+VJSUjBjxgx07doVISEhBu+3Zs0aNG3aVGt9TT+TxHZQoiQm1aFDB437devWhUgk0lpfXmFhIZycnAx+nsDAQAQGBlYrRn18fHwqjVWXiuJnjKG4uBiOjo7VjqmoqAgODg5GL0Ebm5ubm9a5k0ql8PDwqNY5NZWIiAhERUVVaR9976Mx3p+qfv6JaVHVKzG7rl27IiIiAocPH0bHjh3h5OSEESNGAAC2bNmC2NhY+Pn5wdHREeHh4Zg8eTIKCgo0jqGrGjAkJAS9e/fGnj170KZNGzg6OqJp06ZYvXq1UeMfPnw4XFxccO7cOcTGxsLV1RU9evQAwKtvx4wZg2+++Qbh4eGQSqVYt24dAOCPP/5Ajx494OrqCicnJ3Ts2BG7du3SOLaqenDfvn0YMWIE6tatCycnJ5SUlOiMpbi4GJMmTULr1q3h7u4OLy8vREdH4+eff9baVhXbhg0bEB4eDicnJ7Rq1Qo7d+7U2nbXrl1o3bo1pFIpQkNDjVoizMzMxOjRoxEYGAh7e3uEhoZixowZKC0t1dhu2bJlaNWqFVxcXODq6oqmTZti6tSpAPh5evnllwEA3bp1U1efrl271igxVvQ+6nt/lEol5s+fj6ZNm0IqlaJevXoYOnQobt26pXFsfZ9/YhmoREksQkZGBgYPHowPPvgAc+bMgUjEf8NdunQJcXFxmDBhApydnXHx4kV89tlnOHHihFb1rS5nzpzBpEmTMHnyZPj4+ODbb7/FyJEj0bBhQzzzzDOV7s8Y0/rCBgCxWKyRmGUyGfr27YvRo0dj8uTJGvvs2LEDR44cwSeffAJfX1/Uq1cPhw4dQkxMDFq2bIlVq1ZBKpVi6dKl6NOnD77//nsMGjRI4/lGjBiB559/Hhs2bEBBQQEkEonOeEtKSnD//n289957CAgIgEwmw2+//YYBAwZgzZo1GDp0qMb2u3btwsmTJzFz5ky4uLhg/vz5eOGFF5CamooGDRoAAH7//Xf069cP0dHR2Lx5MxQKBebPn487d+5Uev4qk5mZiXbt2kEkEuGTTz5BWFgY/vzzT8yePRvXr1/HmjVrAACbN29GQkICxo4diwULFkAkEuHy5ctISUkBADz//POYM2cOpk6diiVLlqBNmzYAgLCwsEpjUCgUWu+xIAgQi8Ua63S9jydPngSg+/15++23sWLFCowZMwa9e/fG9evX8fHHH+PgwYP4559/4O3trT52RZ9/YiEYIbVo2LBhzNnZWWNdly5dGAD2+++/691XqVQyuVzODh06xACwM2fOqB+bPn06K/9xDg4OZg4ODuzGjRvqdUVFRczLy4uNHj260lgBVLhs2LBB4zUBYKtXr9Z5DHd3d3b//n2N9R06dGD16tVjeXl56nWlpaUsIiKCBQYGMqVSyRhjbM2aNQwAGzp0aKXx6lJaWsrkcjkbOXIke+qpp7Ri8/HxYbm5uep1mZmZTCQSsblz56rXtW/fnvn7+7OioiL1utzcXObl5aV1zisTHBzMnn/+efX90aNHMxcXF433iDHGFixYwACwf//9lzHG2JgxY5iHh4feY2/dupUBYAcOHDAoFtW51bWIxWKNbSt6Hyt6fy5cuMAAsISEBI31f/31FwPApk6dql5n6OefmA/9bCEWwdPTE927d9daf/XqVbz22mvw9fWFWCyGRCJBly5dAAAXLlyo9LitW7dG/fr11fcdHBzQuHFj3Lhxw6C4Bg4ciJMnT2otcXFxWtu++OKLOo/RvXt3eHp6qu8XFBTgr7/+wksvvQQXFxf1erFYjCFDhuDWrVtITU016Ni6bN26FZ06dYKLiwvs7OwgkUiwatUqneerW7ducHV1Vd/38fFBvXr11OenoKAAJ0+exIABA+Dg4KDeztXVFX369DE4pors3LkT3bp1g7+/P0pLS9VLr169AACHDh0CALRr1w4PHz7Eq6++ip9//hn37t2r8XOrrF+/Xuv9/euvv7S2K/8+llX+/Tlw4AAAaLWObteuHcLDw/H7779rrK/o808sA1W9Eovg5+entS4/Px+dO3eGg4MDZs+ejcaNG8PJyQk3b97EgAEDUFRUVOlx69Spo7VOKpUatC/AGx8Z0tDDyckJbm5uOh8r/9oePHgAxpjO1+zv7w8AyM7O1nuMimzfvh0DBw7Eyy+/jPfffx++vr6ws7PDsmXLdF6brez8PHjwAEqlEr6+vlrb6VpXVXfu3MGvv/5aYVWyKiEOGTIEpaWlWLlyJV588UUolUq0bdsWs2fPRkxMTI1iCA8PN+g91vcelH9M9f5V9B6X/6Fm6PtLzIMSJbEIuloI7t+/H7dv38bBgwfVpUgAePjwYS1GZhh9LRzLP+bp6QmRSISMjAytbW/fvg0AGtevKjt+WRs3bkRoaCi2bNmisU9FjX8q4+npCUEQkJmZqfWYrnVV5e3tjZYtW+LTTz/V+bjqhwMAvPHGG3jjjTdQUFCAw4cPY/r06ejduzf+++8/BAcH1ziWylTlPVb9AMnIyNBqjX379u1qv7/EPKjqlVgs1ZeHVCrVWL98+XJzhGM0zs7OaN++PbZv365RslUqldi4cSMCAwPRuHHjah1bEATY29trfPFmZmbqbPVqaKzt2rXD9u3bUVxcrF6fl5eHX3/9tVrHLKt37944f/48wsLCEBUVpbWUTZRlY+rVqxemTZsGmUyGf//9F8Djz4mhtQWmpKpG3bhxo8b6kydP4sKFC+pW0cQ6UImSWKyOHTvC09MT8fHxmD59OiQSCTZt2oQzZ87UWgx37tzB8ePHtda7ubnVqEP63LlzERMTg27duuG9996Dvb09li5divPnz+P777+vdgmjd+/e2L59OxISEvDSSy/h5s2bmDVrFvz8/HDp0qVqHXPWrFno2bMnYmJiMGnSJCgUCnz22WdwdnbG/fv3q3VMlZkzZyIpKQkdO3bEuHHj0KRJExQXF+P69evYvXs3vvnmGwQGBuLNN9+Eo6MjOnXqBD8/P2RmZmLu3Llwd3dH27ZtAfD+kACwYsUKuLq6wsHBAaGhoTqrl8s6f/68zpbNYWFh1R5YokmTJnjrrbfw1VdfQSQSoVevXupWr0FBQXj33XerdVxiHpQoicWqU6cOdu3ahUmTJmHw4MFwdnZGv379sGXLFnXzf1P78ccf8eOPP2qt79SpE/74449qH7dLly7Yv38/pk+fjuHDh0OpVKJVq1b45Zdf0Lt372of94033kBWVha++eYbrF69Gg0aNMDkyZNx69YtzJgxo1rHjImJwY4dO/DRRx9h0KBB8PX1RUJCAoqKiqp9TBU/Pz/8/fffmDVrFj7//HPcunULrq6uCA0NRc+ePdWNZzp37oy1a9fihx9+wIMHD+Dt7Y2nn34a69evVyez0NBQLF68GF9++SW6du0KhUKBNWvWVDrc4BtvvKFz/cqVKzFq1Khqv7Zly5YhLCwMq1atwpIlS+Du7o6ePXti7ty5lSZvYlkExhgzdxCEEEKIpaJrlIQQQogelCgJIYQQPShREkIIIXpQoiSEEEL0oERJCCGE6EGJkhBCCNHjietHqVQqcfv2bbi6utKwUYQQ8gRjjCEvLw/+/v56pzZ74hLl7du3ERQUZO4wCCGEWIibN29qjclb1hOXKFVTCt28ebPC2R4MIZfLsW/fPsTGxlY484EloXhNy9riBawvZorXtJ7EeHNzcxEUFKQx1ZwuT1yiVFW3urm51ThRqqZWspYPFcVrOtYWL2B9MVO8pvUkx1vZZTizN+ZZunQpQkND4eDggMjISBw5ckTv9kuWLEF4eDgcHR3RpEkTrF+/vpYiJYQQ8iQya4lyy5YtmDBhApYuXYpOnTph+fLl6NWrF1JSUjRmpVdZtmwZpkyZgpUrV6Jt27Y4ceIE3nzzTXh6ehpltnVCCCGkPLOWKBcuXIiRI0di1KhRCA8Px+LFixEUFIRly5bp3H7Dhg0YPXo0Bg0ahAYNGuCVV17ByJEj8dlnn9Vy5IQQQp4UZitRymQyJCcnY/LkyRrrY2NjcezYMZ37lJSUwMHBQWOdo6MjTpw4AblcrrOeuqSkRGN299zcXAC8flsul1c7ftW+NTlGbaJ4Tcva4gWsL2ZTxcsYg0KhgEKhgDEnUyotLYWdnR3y8/NhZ2f5zUFsLV5BECAWiyEWiyu8BmnoZ8ls02zdvn0bAQEBOHr0KDp27KheP2fOHKxbtw6pqala+0ydOhVr1qzBzp070aZNGyQnJ+P5559HVlYWbt++DT8/P619EhMTdc6Z991338HJycm4L4oQYlVEIhE8PDzg6OhI/aptEGMMhYWFyMnJgVKp1Hq8sLAQr732GnJycvQ27jT7z4byH07GWIUf2I8//hiZmZno0KEDGGPw8fHB8OHDMX/+fIjFYp37TJkyBRMnTlTfVzUHjo2NrXGr16SkJMTExFhNCzGK13SsLV7A+mI2drxKpRLXrl2DWCxG3bp1IZFIjJosGWMoKCiAs7OzVSRhW4uXMQa5XI67d++iXr16CA0N1RpUQFXDWBmzJUpvb2+IxWJkZmZqrM/KyoKPj4/OfRwdHbF69WosX74cd+7cgZ+fH1asWAFXV1d4e3vr3EcqlUIqlWqtl0gk1f5ny8sDunUT4/btHrh8ufrHMYeavG5zoHhNz9piNla8xcXFYIwhICDAJLVLSqUScrkcjo6Oekd9sRS2Gq+9vT1u3LgBxpjW58bQz5HZzoa9vT0iIyORlJSksT4pKUmjKlYXiUSCwMBAiMVibN68Gb17967VN9bREUhOFiEjwwUG/iAhhFgoa0gKpPqM8f6atep14sSJGDJkCKKiohAdHY0VK1YgLS0N8fHxAHi1aXp6urqv5H///YcTJ06gffv2ePDgARYuXIjz589j3bp1tRq3nR3g4sKQny/g4UPA17dWn54QQkgtMmuiHDRoELKzszFz5kxkZGQgIiICu3fvRnBwMAAgIyMDaWlp6u0VCgW++OILpKamQiKRoFu3bjh27BhCQkJqPXZ3dyA/H8jJqfWnJoQQUovM3pgnISEBCQkJOh9bu3atxv3w8HCcOnWqFqKqnLs7kJ4O5ORY/kVvQgjRp2vXrmjVqpXOHgLEAoaws1bu7rxXzcOH5o2DEPLkEARB7zJ8+PBqHXf79u2YOXNmjWIbPny4zph69uxZo+NaArOXKK2Vhwf/S415CCG1JSMjQ317y5Yt+OSTTzT6nDs6OmpsX9FALOV5eXlBqVQa3F2iIj179sSaNWs01unqdaAvPkNjNuRYxkIlympyd+d/Hz6kqldCbAVjQEFB7S+GDvvi6+urXtzd3SEIgvp+cXExPDw88MMPP6Br165wcHDAxo0bkZ2djVdffRWBgYFwcnJCixYt8P3332sct2vXrnj33XfV90NCQjBnzhyMGDECrq6uqF+/PlasWFFpfFKpVCNGX19feHp6qh8XBAHffPMN+vXrB2dnZ8yePRuJiYlo3bo1Vq9ejQYNGkAqlYIxhrS0NPTr1w8uLi5wc3PDwIEDcefOHfWxZsyYgc6dO2vtZwqUKKuJql4JsT2FhYCLi3EWNzcRAgM94OYmqnTbwkLjvYYPP/wQ48aNw4ULF/Dcc8+huLgYkZGR2LlzJ86fP4+33noLQ4YMwV9//aX3OF988QWioqJw6tQpJCQk4O2338bFixdrHN/06dPRr18/nDt3DiNGjAAAXL58GT/88AO2bduG06dPAwD69++P+/fv49ChQ0hKSsKVK1cwaNAgjWNdu3YNW7du1djPFKjqtZpUJUqqeiWEWJIJEyZgwIABGuvee+899e2xY8diz5492Lp1K9q3b1/hceLi4tQNLT/88EMsWrQIBw8eRNOmTSvcZ+fOnXBxcdFY9+GHH+Ljjz9W33/ttdfUCVJFJpNhw4YNqFu3LgDen/7s2bO4du0agoKCAPBJMZo3b46TJ0+ibdu26v3Wr19f4SA1xkKJsppU1yip6pUQ2+HkxLt9GYPqmp+bm1ulnd6NOTBQVFSUxn2FQoF58+Zhy5YtSE9PV08U4ezsrPc4LVu2VN9WVfFmZWXp3adbt25asz95eXnpjQ8AgoOD1UkSAC5cuICgoCB1kgSAZs2awcPDAxcuXFAnyqCgII39TIUSZTU9vkZp1jAIIUYkCEAl+cNgSiWgUPDj1ebgP+UT4BdffIFFixZh8eLFaNGiBZydnTFhwgTIZDK9xynfMEYQBJ0Di5d/7oYNG1YpPl3rKhrzu/z62prYghJlNamuUVLVKyHEkh05cgT9+vXD4MGDAfCS7qVLlxAeHm7myCrWrFkzpKWl4ebNm+pSZUpKCnJycswSNyXKaqKqV0KINWjYsCG2bduGY8eOwdPTEwsXLkRmZqZJEk5JSYnWRBd2dnYVTlpRkWeffRYtW7bE66+/jsWLF6O0tBQJCQno0qWLzqpbU6NWr9WkqnqlIewIIZbs448/Rps2bfDcc8+ha9eu8PX1Rf/+/U3yXHv27IGfn5/G8vTTT1f5OIIgYMeOHfD09MQzzzyDZ599Fg0aNMCWLVtMELUB8Zhr4mZzyc3Nhbu7e6UTdVbm3Dk5WraUwNOT4f59yy9VyuVy7N69G3FxcVYxpRLFa3rWFrOx4y0uLsa1a9cQGhoKBwcHI0SoqSqNeSyBrcar7302NB9Y/tmwUGVLlE/WTw1CCHmyUKKsJtU1SqVSMFpzckIIIZaHEmU1OTgAdnYKANRFhBBCbBklymri/a1KAVCiJIQQW0aJsgacneUAqOUrIYTYMkqUNaBKlFSiJIQQ20WJsgacnChREkKIraNEWQNU9UoIIbaPEmUNUNUrIYTYPkqUNeDkRK1eCSHE1lGirAGqeiWE1CZBEPQuw4cPr/axGzRooDWXpC4hISE6n3vevHnVfm5LR7OH1AA15iGE1KaMjAz17S1btuCTTz5Bamqqep2jo2OtxDFz5ky8+eabGutcXV11bssYg0KhgJ2dZrqRyWSwt7ev8nNXd7+aoBJlDVCJkhAbwxhQUFD7i4EDRvv6+qoXd3d3CIKgse7w4cOIjIyEg4MDGjRogBkzZqC0tFS9f2JiIurXrw+pVAp/f3+MGzcOANC1a1fcuHEDU6dOhVgs1jlpclmurq4az+vr66uefPngwYMQBAF79+5FVFQUpFIpjhw5gq5du2LMmDGYOHEivL29ERMTAwA4dOgQ2rVrB6lUCj8/P0yePFkj5or2q01mT5RLly5Vj+oeGRmJI0eO6N1+06ZNaNWqFZycnODn54c33ngD2dnZtRStJmrMQ4iNKSwEXFyMsojc3OARGAiRm1vl2xcW1jj0vXv3YvDgwRg3bhxSUlKwfPlyrF27Fp9++ikA4Mcff8SiRYuwfPlyXLp0CTt27ECLFi0AANu3b0dgYCCmTp2K9PR0jZJrdX3wwQeYO3cuLly4gJYtWwIA1q1bBzs7Oxw9ehTLly9Heno64uLi0LZtW5w5cwbLli3DqlWrMHv2bI1jld+v1jEz2rx5M5NIJGzlypUsJSWFjR8/njk7O7MbN27o3P7IkSNMJBKxL7/8kl29epUdOXKENW/enPXv39/g58zJyWEAWE5OTo1il8lk7NNPjzCAscaNa3SoWiGTydiOHTuYTCYzdygGoXhNz9piNna8RUVFLCUlhRUVFT1emZ/PGC/f1e6Sn1/l+NesWcPc3d3V9zt37szmzJmjsc2GDRuYn58fY4yxL774gjVu3LjC8xccHMzmzJnDFAqF3ucNDg5m9vb2zNnZWWM5cOAAY4yxAwcOMABsx44dGvt16dKFtW7dWmPd1KlTWZMmTZhSqVSvW7JkCXNxcVHHoWs/xhhTKBTswYMHlcar831+xNB8YNZrlAsXLsTIkSMxatQoAMDixYuxd+9eLFu2DHPnztXa/vjx4wgJCVFXF4SGhmL06NGYP39+rcatQlWvhNgYJycYazqgKs3v6ORU4+dLTk7GyZMn1SVIAFAoFCguLkZhYSFefvllLF68GA0aNEDPnj0RFxeHPn36aF07NMT777+v1XAoICBA435UVJTWfuXXXbhwAdHR0RpVvZ06dUJ+fj5u3bqF+vXrV3is2mS2RCmTyZCcnIzJkydrrI+NjcWxY8d07tOxY0dMmzYNu3fvRq9evZCVlYUff/wRzz//fIXPU1JSgpKSEvX93NxcAHwSWLlcXu345XJ5mapXBrm8tJI9zEv1WmvymmsTxWt61hazseOVy+VgjEGpVEKpVD5+wEgNYhhjgEIB5uQEZSXX/NRlyypQxVz2b2JiIl544QWtbe3t7REQEIALFy4gKSkJv//+OxISEvD555/jwIEDGhNhq86JPnXq1EGDBg10xqTa19HRUes4Tk5OGuvKvwaAJ/fycZTfT/W4IfEqlUowxiCXyyEWizUeM/SzZLZEee/ePSgUCvj4+Gis9/HxQWZmps59OnbsiE2bNmHQoEEoLi5GaWkp+vbti6+++qrC55k7dy5mzJihtX7fvn1wquGvOCcnfvpKSgTs2LEH9vb6P1yWICkpydwhVAnFa3rWFrOx4rWzs4Ovry/y8/Mhk8mMckxd8vLyTHLc4uJiMMbUP/5btmyJ8+fPY/To0Vrb5pcpJXft2hVdu3bF0KFD0a5dOxw/fhytWrWCnZ0dFApFpfEqlUoUFxern7e8wkfXW/Py8jRK0qWlpZDJZBr7hYWF4ddff0VOTo66VLl//364urrC1dUVubm5Ovcrq7J4ZTIZioqKcPjwYY1GQmVjrYzZu4eUb13FGKuwxVVKSgrGjRuHTz75BM899xwyMjLw/vvvIz4+HqtWrdK5z5QpUzBx4kT1/dzcXAQFBSE2NhZubm7Vjlsul2Pv3iQIAgNjAqKje6JczrcocrkcSUlJiImJ0fj1aKkoXtOztpiNHW9xcTFu3rwJFxcXODg4GCFCTYwx5OXlwdXVtdJWpNXh4OAAQRDU32OJiYno27cvGjRogJdeegkikQhnz57F+fPnMWvWLKxduxYKhQLt27eHk5MTduzYAUdHRzRr1gxubm4IDQ3FsWPHMHToUDg4OMDb21vn84pEIsjlcq0k4+TkBDc3N3UBxNXVVeM71s7ODvb29hrrJkyYgG+++QYfffQR3nnnHaSmpuKzzz7Du+++Cw8Pjwr3Aww/v8XFxXB0dMQzzzyj9T5XlHzLM1ui9Pb2hlgs1io9ZmVlaZUyVebOnYtOnTrh/fffB8B/QTk7O6Nz586YPXs2/Pz8tPaRSqWQSqVa6yUSSY3/2UQiwN2dt3otKJDACr5rjPK6axPFa3rWFrOx4lUoFBAEASKRqPJriNWgqg5UPYexqY6p+turVy/s3LkTM2fOxOeffw6JRIKmTZti1KhREIlE8PLywrx58/Dee+9BoVCgRYsW+PXXX1G3bl0AwIwZM/DWW2+hcePGKCkpUVdt6jJ9+nRMnz5dY93o0aPxzTffaMRV/nWXPxdBQUHYvXs33n//fTz11FPw8vLCyJEj8fHHH2tsp+scGnp+RSIRBEHQ+bkx9HNktkRpb2+PyMhIJCUladSpJyUloV+/fjr3KSws1LrwrKpz1vemmpIqUVIXEUJIbRo+fLhWg5rnnnsOzz33nM7t+/fvj/79+1d4vA4dOuCPP/6otPHR9evX9cbVtWtXnd/HBw8e1Ll9ly5dcOLEiQqPV9F+tcms/SgnTpyIb7/9FqtXr8aFCxfw7rvvIi0tDfHx8QB4tenQoUPV2/fp0wfbt2/HsmXLcPXqVRw9ehTjxo1Du3bt4O/vb5bX4O7O/1LLV0IIsU1mvUY5aNAgZGdnY+bMmcjIyEBERAR2796N4OBgAHy4prS0NPX2w4cPR15eHr7++mtMmjQJHh4e6N69Oz777DNzvQS4uzMAApUoCSHERpm9MU9CQgISEhJ0PrZ27VqtdWPHjsXYsWNNHJXhqERJCCG2zexD2Fm7Rw2zqERJCCE2ihJlDfGqV0qUhFgrczUEJLXDGO8vJcoaoqpXQqyTqmuAoZ3OiXVSvb816VJk9muU1o6qXgmxTmKxGB4eHsjKygLAO8wbc2AApVIJmUyG4uJik/SjNDZbi5cxhsLCQmRlZcHDw0Nr+LqqoERZQ1T1Soj18vX1BQB1sjQmxhiKiorg6OhokpF5jM1W4/Xw8FC/z9VFibKGqOqVEOslCAL8/PxQr149ow8OL5fLcfjwYTzzzDNWMfKRLcYrkUhqVJJUoURZQ6pESSVKQqyXWCw2yhdq+WOWlpbCwcHBKhIPxVsxy6+ItnB0jZIQQmwbJcoaUl2jpKpXQgixTZQoa0hV9ZqXBzyab5QQQogNoURZQ6pECQAGTm1GCCHEilCirCF7e+DRPKV0nZIQQmwQJUojoJavhBBiuyhRGoGq5Ss16CGEENtDidIIqIsIIYTYLkqURkBVr4QQYrsoURoBVb0SQojtokRpBFSiJIQQ20WJ0gjoGiUhhNguSpRGQFWvhBBiuyhRGgFVvRJCiO2iRGkEVKIkhBDbRYnSCOgaJSGE2C6zJ8qlS5ciNDQUDg4OiIyMxJEjRyrcdvjw4RAEQWtp3rx5LUasjapeCSHEdpk1UW7ZsgUTJkzAtGnTcOrUKXTu3Bm9evVCWlqazu2//PJLZGRkqJebN2/Cy8sLL7/8ci1HromqXgkhxHaZNVEuXLgQI0eOxKhRoxAeHo7FixcjKCgIy5Yt07m9u7s7fH191cvff/+NBw8e4I033qjlyDWVrXplzJyREEIIMTY7cz2xTCZDcnIyJk+erLE+NjYWx44dM+gYq1atwrPPPovg4OAKtykpKUFJSYn6fu6jSSPlcjnkcnk1Iod6f9VfPs2WBAoF8PChHC4u1T6syZSN1xpQvKZnbTFTvKb1JMZr6L4CY+YpA92+fRsBAQE4evQoOnbsqF4/Z84crFu3DqmpqXr3z8jIQFBQEL777jsMHDiwwu0SExMxY8YMrfXfffcdnFQTSdYQY8BLL/WBQiHCqlV7UadOsVGOSwghxHQKCwvx2muvIScnB25ubhVuZ7YSpYogCBr3GWNa63RZu3YtPDw80L9/f73bTZkyBRMnTlTfz83NRVBQEGJjY/WemMrI5XIkJSUhJiYGEokEHh4CsrOBNm26w8xti3QqH6+lo3hNz9pipnhN60mMV1XDWBmzJUpvb2+IxWJkZmZqrM/KyoKPj4/efRljWL16NYYMGQJ7e3u920qlUkilUq31EonEKB8G1XE8PIDsbKCgQAJL/owZ63XXForX9KwtZorXtJ6keA3dz2yNeezt7REZGYmkpCSN9UlJSRpVsbocOnQIly9fxsiRI00ZYpVQy1dCCLFNZq16nThxIoYMGYKoqChER0djxYoVSEtLQ3x8PABebZqeno7169dr7Ldq1Sq0b98eERER5ghbJ+pLSQghtsmsiXLQoEHIzs7GzJkzkZGRgYiICOzevVvdijUjI0OrT2VOTg62bduGL7/80hwhV4hKlIQQYpvM3pgnISEBCQkJOh9bu3at1jp3d3cUFhaaOKqqo2HsCCHENpl9CDtbQVWvhBBimyhRGglVvRJCiG2iRGkkVPVKCCG2iRKlkVDVKyGE2CZKlEZCVa+EEGKbKFEaCZUoCSHENlGiNBK6RkkIIbaJEqWRUNUrIYTYJkqURqKqei0qAmQy88ZCCCHEeChRGknZGbuoVEkIIbaDEqWRiMWPkyVdpySEENtBidKIqOUrIYTYHkqU1XH/PsSjRiF6+nSAMfVqatBDCCG2x+yzh1glJycIGzagHmOQ370LBAQAoC4ihBBii6hEWR0ODkBQEABAuHJFvZqqXgkhxPZQoqwmFhbGb5RJlFT1SgghtocSZTWpEiWVKAkhxLZRoqwuHYmSrlESQojtoURZTaxBA36Dql4JIcSmVTlRlpaWws7ODufPnzdFPFaDql4JIeTJUOVEaWdnh+DgYCgUClPEYz1UifL+feD+fQBUoiSEEFtUrarXjz76CFOmTMH9RwniieTsjGJPT377UamSrlESQojtqdaAA//73/9w+fJl+Pv7Izg4GM7OzhqP//PPP0YJztLl+/nB4cEDnijbtqWqV0IIsUHVSpT9+/c3chjWqcDPD94pKcDlywCo6pUQQmxRtRLl9OnTjRbA0qVL8fnnnyMjIwPNmzfH4sWL0blz5wq3LykpwcyZM7Fx40ZkZmYiMDAQ06ZNw4gRI4wWk6EKfX35jXKJMjcXUCoBEbUpJoQQq1ejsV6Tk5Nx4cIFCIKAZs2a4amnnqrS/lu2bMGECROwdOlSdOrUCcuXL0evXr2QkpKC+vXr69xn4MCBuHPnDlatWoWGDRsiKysLpaWlNXkZ1Zbv58dvPEqUqqpXxniyVCVOQggh1qtaiTIrKwuvvPIKDh48CA8PDzDGkJOTg27dumHz5s2oW7euQcdZuHAhRo4ciVGjRgEAFi9ejL1792LZsmWYO3eu1vZ79uzBoUOHcPXqVXh5eQEAQkJCqvMSjKKgXIlSKuXDwBYX8+pXSpSEEGL9qpUox44di9zcXPz7778IDw8HAKSkpGDYsGEYN24cvv/++0qPIZPJkJycjMmTJ2usj42NxbFjx3Tu88svvyAqKgrz58/Hhg0b4OzsjL59+2LWrFlwdHTUuU9JSQlKSkrU93NzcwEAcrkccrncoNeri1wuf5wo79yB/P59wNUV7u52KC4WcPeuHP7+1T680alea01ec22ieE3P2mKmeE3rSYzX0H2rlSj37NmD3377TZ0kAaBZs2ZYsmQJYmNjDTrGvXv3oFAo4OPjo7Hex8cHmZmZOve5evUq/vjjDzg4OOCnn37CvXv3kJCQgPv372P16tU695k7dy5mzJihtX7fvn1wcnIyKNYKubigxM0N0txc/LF+PXJDQ2Fn1x2AK/bu/Qvp6dk1O74JJCUlmTuEKqF4Tc/aYqZ4TetJirewsNCg7aqVKJVKJSQSidZ6iUQCpVJZpWMJgqBxnzGmta7s8wqCgE2bNsH90QXBhQsX4qWXXsKSJUt0liqnTJmCiRMnqu/n5uYiKCgIsbGxcHNzq1KsZcnlciQlJcGuaVPgxAl09vMDi4tDQIAY6elAeHgHxMWxyg9US1TxxsTE6HzvLA3Fa3rWFjPFa1pPYryqGsbKVCtRdu/eHePHj8f3338P/0f1i+np6Xj33XfRo0cPg47h7e0NsVisVXrMysrSKmWq+Pn5ISAgQJ0kASA8PByMMdy6dQuNGjXS2kcqlUIqlWqtl0gkRvkwCGFhwIkTsLt+HZBI1Ncl8/PtYImfNWO97tpC8ZqetcVM8ZrWkxSvoftVqwPD119/jby8PISEhCAsLAwNGzZEaGgo8vLy8NVXXxl0DHt7e0RGRmoVm5OSktCxY0ed+3Tq1Am3b99Gfn6+et1///0HkUiEwMDA6ryUGlPPS0l9KQkhxCZVq0QZFBSEf/75B0lJSbh48SIYY2jWrBmeffbZKh1n4sSJGDJkCKKiohAdHY0VK1YgLS0N8fHxAHi1aXp6OtavXw8AeO211zBr1iy88cYbmDFjBu7du4f3338fI0aMqLAxj6lVlChpdB5CCLENVU6UpaWlcHBwwOnTpxETE4OYmJhqP/mgQYOQnZ2NmTNnIiMjAxEREdi9ezeCg4MBABkZGUhLS1Nv7+LigqSkJIwdOxZRUVGoU6cOBg4ciNmzZ1c7hhpr2JD/LdeXkhIlIYTYhionSmPPHpKQkICEhASdj61du1ZrXdOmTS2qVZa6RHnrFlBUBA8PXrKlqldCCLENNHtITdWp87gYee0aVb0SQoiNodlDakoQePVrcjJw+TLc3ZsBoERJCCG2gmYPMYawMHWi9GjCV1HVKyGE2IZqNeYBgBEjRiAoKMjoAVmlMg163Nvxm1SiJIQQ21Dla5R2dnZYsGCB0Rrz2IQyiZKuURJCiG2pVmOeHj164ODBg0YOxYrpSJQ5OXy6LUIIIdatWtcoe/XqhSlTpuD8+fOIjIzUaszTt29fowRnNVSJ8sYNuDvKANhDLgeKioCajrtOCCHEvKqVKN9++20AfEDy8gRBePKqZX19eUYsLIRL9g2IRI2gVPJSJSVKQgixbtWqelUqlRUuT1ySBHgXkUcDDwhX6DolIYTYkiolyri4OOSU6ffw6aef4mGZbJCdnY1mzZoZLTirUrblKw1jRwghNqNKiXLv3r0oKSlR3//ss880RucpLS1Famqq8aKzJhU06CGEEGLdqpQoWblmnOXvP9GoREkIITapWtcoiQ6qRHnlCl2jJIQQG1KlRCkIAgRB0FpH8DhRXr0KTzfeoImqXgkhxPpVqXsIYwzDhw+HVCoFABQXFyM+Pl7dj7Ls9csnTkAAYG8PyGQIEd8EEEIlSkIIsQFVSpTDhg3TuD948GCtbYYOHVqziKyVWAw0aABcvIiQ0ssAQqhESQghNqBKiXLNmjWmisM2NGwIXLyIgKLLAJ6lEiUhhNgAasxjTI+uU/rkXQZAjXkIIcQWUKI0pkeJ0usBT5RU9UoIIdaPEqUxPUqUbnevAKASJSGE2AJKlMb0aLxXp9tXIEBJiZIQQmwAJUpjCg4GxGKISorghwyqeiWEEBtAidKYJBIgJAQA0BCXUVAAyOXmDYkQQkjNmD1RLl26FKGhoXBwcEBkZCSOHDlS4bYHDx5Ujw5Udrl48WItRlyJR9cpG4Ia9BBCiC0wa6LcsmULJkyYgGnTpuHUqVPo3LkzevXqhbS0NL37paamIiMjQ700atSoliI2wKNE2UxCiZIQQmyBWRPlwoULMXLkSIwaNQrh4eFYvHgxgoKCsGzZMr371atXD76+vupFLBbXUsQGeJQoG9tRy1dCCLEFVRqZx5hkMhmSk5MxefJkjfWxsbE4duyY3n2feuopFBcXo1mzZvjoo4/QrVu3CrctKSnRGIM2NzcXACCXyyGvwQVE1b7ljyEEB8MOQBjjJcrs7FLI5eafjqyieC0VxWt61hYzxWtaT2K8hu5rtkR57949KBQK+Pj4aKz38fFBZmamzn38/PywYsUKREZGoqSkBBs2bECPHj1w8OBBPPPMMzr3mTt3LmbMmKG1ft++fXBycqrx60hKStK475Kejh4A6ssuAWDYv/8fFBVl1Ph5jKV8vJaO4jU9a4uZ4jWtJynewsJCg7YzW6JUKT9NF2Oswqm7mjRpgiZNmqjvR0dH4+bNm1iwYEGFiXLKlCmYOHGi+n5ubi6CgoIQGxsLNze3asctl8uRlJSEmJgYSCSSxw8UF4ONGwcXZT7q4i7CwtogLs4ySpQ647VQFK/pWVvMFK9pPYnxqmoYK2O2ROnt7Q2xWKxVeszKytIqZerToUMHbNy4scLHpVKpelqwsiQSiVE+DFrHkUiAoCAgLQ0NcRn5+R1hSZ85Y73u2kLxmp61xUzxmtaTFK+h+5mtMY+9vT0iIyO1is1JSUno2LGjwcc5deoU/Pz8jB1ezZTpIkKNeQghxLqZtep14sSJGDJkCKKiohAdHY0VK1YgLS0N8fHxAHi1aXp6OtavXw8AWLx4MUJCQtC8eXPIZDJs3LgR27Ztw7Zt28z5MrQ1bAjs34+GuIzsh+YOhhBCSE2YNVEOGjQI2dnZmDlzJjIyMhAREYHdu3cjODgYAJCRkaHRp1Imk+G9995Deno6HB0d0bx5c+zatQtxcXHmegm6PRrzNQxXcJX6URJCiFUze2OehIQEJCQk6Hxs7dq1Gvc/+OADfPDBB7UQVQ2VqXrd+tC8oRBCCKkZsw9hZ5PoGiUhhNgMSpSm8KjqtQ7ug2XfN3MwhBBCaoISpSk4O0PmzVviStKuoKDAzPEQQgipNkqUJmLXhFe/1s29jJkzzRwMIYSQaqNEaSKixjxRhuEKFi4Ezp0zc0CEEEKqhRKlqTy6Ttmj/mWUlgKjRwNKpZljIoQQUmWUKE3lUcvXjj6X4eIC/Pkn8O23Zo6JEEJIlVGiNJXwcACA9MxJfDWBz0354YfAnTvmDIoQQkhVUaI0lRYtgJgYQCbDsFMT0KYNn8R50iRzB0YIIaQqKFGaiiAA//sfIJFA2LUTmwfvhEgEbNoE/PabuYMjhBBiKEqUptS0KfDuuwCARl+Px/jRxQCAhASguNicgRFCCDEUJUpT++gjwN8fuHoVc+osgL8/cOkSMHeuuQMjhBBiCEqUpubqCixYAABw+GIOVn50AwBPlBcvmjMwQgghhqBEWRteeQXo0gUoKkKv3yYiLg6Qy4H4eIAxcwdHCCFEH0qUtUEQgK+/BsRiCNu3Y9WgfXB0BA4dAh7NSU0IIcRCUaKsLRERwNixAADfT8dixjQZAOC994DsbHMGRgghRB9KlLUpMRHw8QH++w8ThUWIiADu3QN69wbOnjV3cIQQQnShRFmb3N2B+fMBAOI5s7B+zi04OQHHjwNPPQWMGwea6JkQQiwMJcraNngw0LEjUFCApza9hwsXgJde4gOmf/UV0LgxsHo1DaBOCCGWws7cATxxRCJgyRIgMhLYsgX1R4/G1q3d8Ntv/BLmxYvAyJEMO/93DQsGHEODO3/yObqkUt7VRLW4uWne9/UFunUD7O3N/QoJIcSmUKI0h9ated+QpUuBMWOA48fxrMMZnBtyDNe+/xPu/x5DvTNZwJkqHtfbGxg2DBg1io8KRAghpMYoUZrLrFnADz8AKSn82iVjsAPQ6NHDcpE9/la2wTF0xAXHSISGMPg656GeQy7q2OfB0y4XbkIeXJR5cJTnQpJ6DkJGBvDFF3zp3Bl4801er+voaM5XSoh+16/zflL16/MfeoJg7ogI0UCJ0ly8vHjDnhEj+KgDfn5AdDS/fhkdDUmbNig96YD1Yx61iL2g/3DeHqXYNmk3nkldCezeDRw5wpexY/l10eHD+YYKBZCVBdy6xZf09Me3b90CCgt59a2+xdmZJ3c3N92Lqyt/rtJS/Qtj/GKsjr+CTAafv/+GAPDnFIn4IhY/vi0SAfXqAY0a0ZertVEqgb17ea3Krl2PR97Ys4dP3OriYt74njRZWRBPmICoGzeAVq2AkBBzR2RRKFGa0xtv8A9lnTr813S5L/vOnYHkZODYMZ7DMjOBjAz+t+xy7x5w76EdunzRF1On9sWsJbcgWr+Wf+HcuAEsWQLJkiV4zt0ddvn5PFlaODsAHQzduHFjYMAAvkRFGZ40S0v5r5C//gIkEv4e1K8PBAXxHwPWjDHL/PFw7x6wZg3wzTfA1auP13fsCJw4AWzZwt+TbdvUc7qSShQU8B+81f3BuG8fMHQoRHfuIAAAi4wEVq0C+vc3dqScQsE7j9+5U/GSmcn/P59/Hnj9dT5toRmZPVEuXboUn3/+OTIyMtC8eXMsXrwYnTt3rnS/o0ePokuXLoiIiMDp06dNH6iptGmj92E7O+CZZ/QfQiYDPvkE+OwzYM4c4Ny5QGzc+BHcpk7lc3qtXAm2YwcccnL4DmIxH6g9MFB7cXbm4+vJZLqXkhL+j5mbq3vJyQHy8vg/rJ2d/kUk4tvp+KsEkPPwIdzd3SFijP9zKZWai0LBfwj89x8wbx5fgoKAF17gSfPpp/lrVSko4Enxjz/48uefQH6+7pNapw4/VtnkWbcu4On5ePHy4n+dnAx/v00pJ4eXzn76iZfMFApe4q5Xj8de9m+9ehA8PFD3zBkIjo6apfayi0TC+/76+Giey6pgjJ/3pUv55YaSEr7ew4PXdMTHA02aAEePAgMHAhcuAG3b8i/rQYOMdXZsT1oaH/Fr5UrerywqitdSdetm2P4lJcC0afxSDQDWrBlyiovhcfUq/x966y1g4cKq/WgsLuafvRs3gLt3+Q+ju3c1l/v3DR+78/x5/sXWogXw2mt8qV/f8HiMRGDMfKONbtmyBUOGDMHSpUvRqVMnLF++HN9++y1SUlJQX8/JyMnJQZs2bdCwYUPcuXOnSokyNzcX7u7uyMnJgZubW7Vjl8vl2L17N+Li4iCRSKp9HGPatAkYOZJ//ps3B37+GQgL44/Jb9/Gse++Q8eXX4YkMFDnl15qKm9cawm1Lgaf39xcXtW8fTv/W1Dw+LG6dYF+/Xg13h9/AKdOaZem3dx4lbdYzL940tL4MatCIgHz8kKevT1cIiMhatmSj8TUogXQsCH/UaALY/yX89mzmktxMa9peOqpx4uPj+5jZGbyN/qnn4D9+/mPHFMQi3nLan9/ICBAc3Fz419+2dmPqjfuad5W3Vdp0wZ45x0+BnL5Hxl37gCvvgocOMDvjxsHfP45YG9vkf9z+pgkXsZ4FdPixfw9V32eBeFx8unZk/9obNWq4uOkpvLzfOoUv//OO5DPmYP/27cPzx8/DvGj5ImmTYHvvuOfQX0yMoBly3hNwd27hr0Wb+/HP8LKLr6+/G9eHrB5M//xJ5M93q9zZ+D11yHv3x+7jx+v0fk1NB+YNVG2b98ebdq0wbJly9TrwsPD0b9/f8zVMw/VK6+8gkaNGkEsFmPHjh2UKMs4eZLXmNy+zQs8W7cC3btXHG9hId9m+XJewLK3BxYtAt5+27w1d9U6v0VFQFIS/wL5+WfgwQPtberXBzp14qXNp5/mvyjK/2jIyQFu3nycONPS+P3sbH7MBw94YnjwgFcP6WNvz6sQIyL44u3NG3CdPQucOcOTiCH8/B4nzdateQOYn37ib1rZf+HwcF4a6N+fl4qzsvgXV9m/j26zu3eR++AB3JydIaiuD5dfiov59jXt2CuV8sSYkMBLi/o+XKWlvIpE9R0QHQ388APkPj7m/5978AC4fBm4coWXuFu14j+GdPzwNOp3hEzG/1EXLwb+/vvx+mefBSZM4N3NZs/m/8ilpfz8Dh7MGw0GBz/enjFeUh8/nv/z16nDO2737asZ76FDwNChPAFKJDzxTpjAX3NZycnAl1/yhKb6kRYYyKvS69bli7f349uqpU6din9AlvfgAa+K37SJD5D96PPOJBKcGTUKzb/80uSJ0mxVrzKZDMnJyZg8ebLG+tjYWBw7dqzC/dasWYMrV65g48aNmD17dqXPU1JSghJVVQ/4iQH4h1heg1/fqn1rcgxTaN2a/+B8+WUxTp4UITaWYeFCJUaO1Iz3/Hng229F2LRJhJycx19aMhn/sf/770osX66Au7s5XkU1z6+dHdCrF1+WLIFw5AiEnTsBhQIsOhqsY0ftahtVQijLyYlXBTZpov/5GOMl2AcPUJqVhVN79iDK0RF2Fy8C//4LISUFQkEBT4hndPf1YSIR0KgRWIsWfGnZEpBKIZw5A+H0aQinTwP//cdbNGdk8FJzOcq2bcH69YOyXz/tmIOCKgxfLpfjYFISYmJi9H/RKBTAnTsQbt8G0tN5LOnp/P7t2/yXv5cXUKcOWJ06/EvQ2xvMy+vx3+BgXvIEKv9xAQAzZkCIioJ4xAgIf/4J1qYNFGvWqOM2qeJiCMnJwJUrEC5fhnD1KnD1KoQrVyDo+PHFnJz4e9e6NVirVkCrVmDNm0P+6JzqjJcx/iOksJAvRUVAURGER39RVKReL1y7BtGqVfy8A2BSKdjrr0MxZgz/8aWyaBHwzjsQf/IJRD/+CGzYALZlC5QJCVB++CEgEkH89tsQbd8OAFB27w7F6tW8lqDM96FcLuezHSUnQzx6NES//gpMmgTlnj1QfPstULcuhJ9/hujrryE6elT99MroaCjHjgXr37/yJMiY4bUfLi68NfSwYcCtWxBt2QLR999DOHsWOaGhRvker4zZSpS3b99GQEAAjh49io4dO6rXz5kzB+vWrUNqaqrWPpcuXcLTTz+NI0eOoHHjxkhMTKy0RJmYmIgZM2Zorf/uu+/gZCnXlkxAJhNh6dLWOHiQf1HGxl7HsGH/4q+//LB3bwhSU73U2/r4FCAm5ga6d0/DH38EYN265lAoRPDxKcD7759Ew4Y55noZ1k2phNPdu3C7cQOuN27ALS0N9rm5yAsKQm5ICHJDQpAXFASFVKr3MOLiYrhdvw73q1fhfu0a3K9fh9zJCZnt2yOjfXsU16lTSy+o9jllZKDt/PnwuHYNTCRCRvv2eBgWhpwGDZDToAFKPDyM92RKJYIOHED4pk1wvH+/ws2KPT1R4OsLQaGA2/XrsCtbLfgIE4mQ7+8PmYsLxCUlEMtk/O+jRdc+lSn29MS1uDhcj42FrJJfsB6XLqHZ+vWoe+4cAEDu5IRSR0c4ZmdDKRbjwuuv43L//tolRK0XwhCydy+ar14NO5kMJW5uUEilcHpUvaoUi5H+9NO42rs3HjZqpP9YRuZy8ybyAwNrVPVVWFiI1157zXKrXlWJ8tixY4iOjlav//TTT7FhwwZcLDersUKhQIcOHTBy5EjEx8cDgEGJUleJMigoCPfu3atx1WuSIb/GzYgxYNEiEaZMEYExASKREkol/8ews2Po04dh1CglevRgGv8vJ04IeP11MW7cEGBvzzB/vhJvv62s1apYazi/ZVlbvIAVxVxUBPH48RCtXav1EPPz4yW5li3536eeAkJDq/zlKRw4APEHH0B4VPJn9eqBRUSAhYUBDRqAhYWBNWgANGig2XVFoQAuXeI1AKrl9GkIhl6nA69ChJMT7+/8aGGOjo/XubhA2bcv2EsvVW3kLcYg7NsH8dSpEB4lTNawIRTr14NFRWltrvfzcOEC7IYMgfBo9gbm7Q3lm29COXo0L5GagTE+v7m5ufD29rbcqldvb2+IxWJkZmZqrM/KyoKPjoYLeXl5+Pvvv3Hq1CmMGTMGAKBUKsEYg52dHfbt24fu3btr7SeVSiHV8YtdIpEY5cvBWMcxlQ8/5G1KXn2VITdXhNBQhjffFDB8uAA/PwG6hvvt1Ilf4x8xAtixQ8CECWIcPizGqlW8oWJtsvTzW561xQtYQcwSCbBmDUrfeAOpq1cjvLgYojNngNRUCBkZvEry//7v8fYNGwIvvshbPld2PfTiReD994GdO/l9d3dg2jQIY8dCcHAwLLYWLfgyeDBf96iRVunff+Ofo0fR5umnYefmxhOfKvmVuS3oqKYsH3G1B+Xu3RuIi+Pdbq5cgTB+POxU/ZwrfEk6Pg8tW/LuO0uWAF5eEAYNgtjREdVsB21UNfn8Grqf2RKlvb09IiMjkZSUhBdeeEG9PikpCf369dPa3s3NDece/SpSWbp0Kfbv348ff/wRoaGhJo/ZWsXFAadPl2Lz5uOYOLEDpNLKPxyenrwh6Vdf8Tkzt2/nyfOHH3grdIBfmrp2jXeHK7ukpwPt2/NE2769ZXbnI9aHRUfj8oMHaBwXB5FEwq8Pnz3LP5iq5dw53tjms8/4EhTEE+aLL/IGJqpGN3fv8mnvli/npUKxmLdgmz6dNz6pCUEA/PzAevZEhlIJ9txzPKGai0jEW7jWlFQKTJxY8+NYIbP2o5w4cSKGDBmCqKgoREdHY8WKFUhLS1NXrU6ZMgXp6elYv349RCIRIspeuAZQr149ODg4aK0n2gIDgWbN7ld6SaIsQeCt86Ojefe2a9f4d81TT/Hb+mqXzp7l3buaNeMJc/Dgins4EFItzs78w1nm0g3y8niDp23b+N+bN3mrzC+/5B/A/v15C+KFCx93A+rTh/c/pPGRSQXMmigHDRqE7OxszJw5ExkZGYiIiMDu3bsR/Kg5c0ZGBtLS0swZIgGvvTp1ivfR3L6d18Co1KnDL9uUXby8gB07gB9/5D0h3nsPmDyZ1wKNGMEbpRraMpyQKnF15YMUDBrEW47u28eT5i+/8D6ay5c/3rZ1a54wDe2gT55YZv+6SkhIQEJCgs7H1uq4eF9WYmIiEhMTjR8U0eLhwRPf77/zH+INGvA2ExU1vhswgFfbbtnCu2n99RdPnjt28P7EQ4fyH/ft2lV/wBdC9HJ05ANO9OvH+z3t389/6V26xLsaDBlCHz5iELMnSmI9BIH3bzaUuzsfBeutt4B//+VDfK5fzweTmT+fL56eQEwM8NxzfAkIMF385Almb89HrOnZ09yRECtEiZLUiubNgQUL+Fi0u3YB33/PB9F58IA3EPrhB75dixY8YT77rIDiYjHu3OEl2Ozsx6OiqW5nZ/PLTR988LgvOyGEGBslSlKr7O35CGsvvMAHaDl5ko+hvHcvv/Z57hxfFiywA9DboGNu3swTb9u2po2dEPJkokRJzMbO7nGjxRkzeAkxKUmVOBkyMwUIAoOnpwDVyGhlRkiDhwefSezKFd4ad84cYNKkygcbIYSQqqBESSxGnTp83OxXXgFkslJs3ZqEF1+MgYNDxX3QxowBRo/m40V/8AFPtOvW8SpZQggxBvrtTSySIAAuLvJKGyV6evKWtStX8kaOSUl8QoeyA7VYgvv3+Q8APz8+LWNNJ+MghNQeSpTE6gkCMGoUn/GnZUs+EEJcHB9EpMwwv9WmUFRtrtnyDh3iyXvLFt7i9513+DCB5QaaIoRYKEqUxGaEh/P+mmPH8vuLFvHrn0lJvKHQmTPAhQuPh9m7e5e3qC0q4lNOHjkCbNjAp/AbORLo0YMPG+royKuF27XjJVVDE6ZqWsXu3YFbt4BGjYCZM3mf+OPH+fzFU6bw5yeEWC66RklsioMD8L//8b6Zb7zBRxSKjTXOsf/+m5dUO3bkCa9794rHsb1+HXj9dT43KAAMH84HYHBx4aMTjR3L512eN493jfnmGx4zIcTyUImS2KQ+fXgJ8qWXgMaN+bzBvr58eD1nZ+0xqiUSXnrs0YOXJmfN4oMjHD7MS5sZGbxFrYMDT37PPstHPjtyRPu5f/jh8QTabm7Ad9/xwRZUszMFBPABYnbs4LevXuXJfPBgICvL1GeGy8kBfvlFwJ07jrXzhIRYMSpREpsVEMBbw1ZEqeSTrJeU8CRWWbeSBQt4spw7lw8ZeugQ8MwzvCT4ySd8gITRo8VYs4Zv36EDT5IVTWzTrx8vlX70ES9tbtrEq3bnzeMlUFNMOHHvHh8f/KuvgJwcOwhCDH79lWHcOJ78aaYXQrRRiZI8sUQiPnOQm5vhfS/9/HjV7pUrQHw8T2ZJSUDnznZ4880YrFkjgiAA06bx0mhls7+5uvLEdfw4b/Bz/z4f8i8sjF9jzcur+esEeCOi998HQkKA2bN5idLfn4ExATt3ihAby6/xfvXV40k1CCEcJUpCqiEwEFi2DPjvP37NUSxmyMuTwt+f4fffeTKqSomwXTs+StEXX/DZoG7e5K1269fnSbfc/OYGS0vj10NDQniJuKCAT5O2bRtw9Wopvv76d7zzjgKurkBqKp9WLSCA90+9cKF6z0mIraFESUgNhIQAq1YB586V4q23ziA5ubTaszZJJDw5Xr8OrFjBr60+fMhHHAoJ4QMr/Pdf5ccpLgbOnwfefJNfd/36a169HB3Nx9lNTuazu4hEQGBgPhYtUiI9nU9eHx4O5Ofz282a8erYX3+lfp/kyUaJkhAjaNgQiIu7jjp1an4sBwee5FJSeKOfDh14oluxgs8tPGAAr/79+GOePF94gffLbNiQVyM7OvLB5b/9ll+D7daNT4929ChvtavrOqSrK5CQwGd5+f13fkyRiN/u25c/75IlPIkS8qShREmIhRKLecI6doy3ru3Th/fh/OknYPx4Xr27YgVvPXvsGL9uqrqmaW/Pk+Iff/BpGPV1ZSlLEPi227cD167xYQE9PPgUjmPGAEFBfN3Nm8Z5jTdv8r6mYWG8SnjdOj51JCGWhFq9EmLhBAF4+mm+pKTwkl1WFr+WWa+e9uLjw0uWNW3BWr8+8NlnvOS6bh2weDFw+TLw+efAwoW8682ECbzEWxUKBbBvH7/Gu2uXZrXu8OH8muyECbxRE02fRiwBJUpCrEizZjxR1iYXFz7s3ttv88S2aBFw4AAfkm/LFl4SjIzk1bOqJSQEWuP03rkDrF7NS8HXrz9e3707T4o3bvBknJ7OW+jOmsVbFo8fD/j71+ILJqQcSpSEEIOIRLz6t08f4PRpntS+/56PfnTqlOa2UilvjKRKnP/9x6tz5XL+uKcnLz2OHg00afJ4v/HjeX/SBQt4q9v583liHjKEP8YYr14uLOQNnXJyHv/NyeHDAT7zDE/c1CeUGAslSkJIlbVuDaxdywdH2L+fdy25eJEvqam88ZFqEu6yoqN5KfHll3mjo/KkUt7dZvhwXnr9/HN+fXb1amD1aglEor5QKivPgE2b8pGOXn+dl24JqQlKlISQavP1BV57TXOdQsGrUVWJ8+JFPmzg8OF8UAVDlC29Hj/OE+ZPPzF1kpRIAHd33tCo7N/SUn798+JFPuLRRx/xEubgwTw5e3hoP1dBAe9Oc/YsH/bw7Fm+/qOPjDdOsKndv8/Pu1isvdjZ0WTmNUWJkhBiVGIx0KABX+Lian68Dh34AAkZGaXYs+d3vPhiD7i6SiqsWs3N5dtv3MivpR4+zJexY3nijYvjrW1VSfHKFd0zwjz3HDBwIG+4FBBQ89dhCqWl/PquathEfby9gd69efeimBjeDYkYhn5nEEKsgrc34OVVAkdH/dcf3dz4zDG//85LtvPmAc2b8+rgH3/kVbvTp/Nrppcv8yTp68tLj++9x6daGzeOl8J++IFX4y5axJOSoWQy4OBBAWfOeOPOnerPZaqPQgEMG2ZYkgT4OL9r1/J+sXXrAoMG8cZYxhom0ZaZPVEuXboUoaGhcHBwQGRkJI7omo7hkT/++AOdOnVCnTp14OjoiKZNm2LRokW1GC0hxJoEBQEffsivlZ46xUc+evppXg28cCHw22+8NW5GBrB3L6/iHTyYj7+bnMxLs/n5fL/ISD5oQ0WKioCffwaGDuVddGJj7TB9eicEBUlQrx7QtSvvi7psGb/uev9+9V+XKkl+9x2vWv3pJ97NprSUj8xUUMBL1vfv83lXMzN56XrcOD78Yn4+/xHwyis8afbpA6xbJyA31wQj8dsCZkabN29mEomErVy5kqWkpLDx48czZ2dnduPGDZ3b//PPP+y7775j58+fZ9euXWMbNmxgTk5ObPny5QY/Z05ODgPAcnJyahS7TCZjO3bsYDKZrEbHqS0Ur2lZW7yMWV/M5ohXoWBs5UrGvLwY4+VCxkaMYOzuXf54Xh5jW7YwNnAgY87Oj7cBGPPxUTI/vzwmCEqN9WUXPz/GPvyQsYICw2MqLWVsyBC+v50dY9u3V+01KZWMnTjB2OTJjDVqpBmPWKxgcXEKtnlz1WIyB2N8HgzNB2ZNlO3atWPx8fEa65o2bcomT55s8DFeeOEFNnjwYIO3p0RJ8ZqCtcXLmPXFbM54797lCVKVULy8GHv+ecYcHDQTTVAQY+PHM3b4MGNFRTzehw9lLDmZsXXrGPvgA8bi4hgLDtbcLzSUsX37Ko+jtJSxoUNVSY2xH3+s2etSKhk7f56xmTMZa9FCM6G7uDA2bBiPq7S0Zs9jCrWZKM3WmEcmkyE5ORmTJ0/WWB8bG4tjqmnhK3Hq1CkcO3YMs2fPrnCbkpISlJSUqO/nPppDSC6XQ67q1FUNqn1rcozaRPGalrXFC1hfzOaM190d+OYbYOhQAWPGiHH+vIBdu/hjYWEM/fsrMWAAQ1QUU18/VcUpkcjRogUff7esvDwgKUnAe++Jce2a8GjybiU+/1yhc8xghQJ46y0xNmwQQSxm2LhRgb59GWp6Oho3BiZPBiZNkmP16uNIS+uMH36ww40bAtat46My+fkxDByoxGuvKdG6tWX0UTXG58HQfQXGTHGZuXK3b99GQEAAjh49io4dO6rXz5kzB+vWrUNqamqF+wYGBuLu3bsoLS1FYmIiPv744wq3TUxMxIwZM7TWf/fdd3BycqrZiyCEPHEUCgEHDgThwQMHREVlIiQkt0aJo6jIDhs2hOP//i8UjAlwcyvByJHn8cwzt9THVSqBr79+Cvv314dIpMSkScno1Om2cV6QDkolcPGiFw4fDsTRowHIy7NXP+bjU4CoqDuIjLyDiIh7sLev+tQySqVldFkpLCzEa6+9hpycHLjpGS/R7Iny2LFjiI6OVq//9NNPsWHDBly8eLHCfa9du4b8/HwcP34ckydPxtdff41XX31V57a6SpRBQUG4d++e3hNTGblcjqSkJMTExEBiiqnojYziNS1rixewvphtPd7jxwXEx4uRksKz43PPKfHVVwrUrw+MHi3GunW8JLl+vQIvv2z8r+2K4pXJgL17BXz/vQg7dwooLn78q8DRkaFbN4a4OIaePZWoX1/zmAoFb1l89qyAs2cFnDvHl/R0IC6OYfJkJdq1q95rMcbnITc3F97e3pUmSrNVvXp7e0MsFiOz3Iy0WVlZ8PHx0btv6KNp41u0aIE7d+4gMTGxwkQplUohlUq11kskEqP8sxnrOLWF4jUta4sXsL6YbTXezp15y9z58/k4t3v3itC6tQgdOvDRj0QiYNMmAYMGmfZru3y8EgnvezlgAG8t+9tvwO7dfElPF7B7t4DduwFAjObNeTebvDzeT/X8ed4aWJedOwXs3ClCjx58IPyuXQ2r0i0tVY01LEb37nY1+jwYup/ZCr/29vaIjIxEUlKSxvqkpCSNqtjKMMY0SoyEEGKt7O35iEBnz/IRhQoLHyfJjRt530dzcnEB+vfnA9vfvMnH/J0zh8+HKhLx+UwXLeJzoZ48yZOkoyPQrh2fY/Xrr3nXmFOneBcdOzve37V7d36MnTt19zktLQWSkvjgCqo+r6tWiXDihG+tvG6zjswzceJEDBkyBFFRUYiOjsaKFSuQlpaG+Ph4AMCUKVOQnp6O9evXAwCWLFmC+vXro2nTpgB4v8oFCxZg7NixZnsNhBBibE2a8FLTt9/ycW4nTeJD8FkSQeBDErZqBUyZwvts7tsHHDrEB4do2ZI/FhamPZMMwAdKSEzkfVe//Rb480/en7NlS2DqVJ6QjxwBtm7lg0Pcu/d4X29v4IUXFAgOrp3REsyaKAcNGoTs7GzMnDkTGRkZiIiIwO7duxEcHAwAyMjIQFpamnp7pVKJKVOm4Nq1a7Czs0NYWBjmzZuH0aNHm+slEEKISYhEvAT11lvmjsQwXl58AINXXjF8n+BgXsr86CNeEl26lJemX3mFV/mWbZTq7c2rfwcOBLp0ARhTYvfuHOO/EB3MPtZrQkICEhISdD62du1ajftjx46l0iMhhNgYX18+SfiHH/LE+eWXvIRapw7w4ou8NN21K6+qVanNXkJmT5SEEEIIwEuln3zChwy8epVPVG5nAVnKAkIghBBCHnNx4dcqLYUFdPkkhBBCLBclSkIIIUQPSpSEEEKIHpQoCSGEED0oURJCCCF6UKIkhBBC9KBESQghhOjxxPWjVM0qpprAubrkcjkKCwuRm5trFTMZULymZW3xAtYXM8VrWk9ivKo8UNlsk09coszL44PoBgUFmTkSQgghliAvLw/u7u4VPm62iZvNRalU4vbt23B1dYVQg2nJVRNA37x5s0YTQNcWite0rC1ewPpipnhN60mMlzGGvLw8+Pv7QySq+ErkE1eiFIlECAwMNNrx3NzcrOJDpULxmpa1xQtYX8wUr2k9afHqK0mqUGMeQgghRA9KlIQQQogelCirSSqVYvr06ZBKpeYOxSAUr2lZW7yA9cVM8ZoWxVuxJ64xDyGEEFIVVKIkhBBC9KBESQghhOhBiZIQQgjRgxIlIYQQogclympYunQpQkND4eDggMjISBw5csTcIQEAEhMTIQiCxuLr66t+nDGGxMRE+Pv7w9HREV27dsW///5bqzEePnwYffr0gb+/PwRBwI4dOzQeNyTGkpISjB07Ft7e3nB2dkbfvn1x69Yts8Q7fPhwrXPeoUMHs8Q7d+5ctG3bFq6urqhXrx769++P1NRUjW0s6fwaEq8lnV8AWLZsGVq2bKnu5B4dHY3/+7//Uz9uSefXkHgt7fyWN3fuXAiCgAkTJqjXmeUcM1IlmzdvZhKJhK1cuZKlpKSw8ePHM2dnZ3bjxg1zh8amT5/OmjdvzjIyMtRLVlaW+vF58+YxV1dXtm3bNnbu3Dk2aNAg5ufnx3Jzc2stxt27d7Np06axbdu2MQDsp59+0njckBjj4+NZQEAAS0pKYv/88w/r1q0ba9WqFSstLa31eIcNG8Z69uypcc6zs7M1tqmteJ977jm2Zs0adv78eXb69Gn2/PPPs/r167P8/Hz1NpZ0fg2J15LOL2OM/fLLL2zXrl0sNTWVpaamsqlTpzKJRMLOnz/PGLOs82tIvJZ2fss6ceIECwkJYS1btmTjx49XrzfHOaZEWUXt2rVj8fHxGuuaNm3KJk+ebKaIHps+fTpr1aqVzseUSiXz9fVl8+bNU68rLi5m7u7u7JtvvqmlCDWVTzyGxPjw4UMmkUjY5s2b1dukp6czkUjE9uzZU6vxMsa/aPr161fhPuaMNysriwFghw4dYoxZ/vktHy9jln1+VTw9Pdm3335r8ee3fLyMWe75zcvLY40aNWJJSUmsS5cu6kRprnNMVa9VIJPJkJycjNjYWI31sbGxOHbsmJmi0nTp0iX4+/sjNDQUr7zyCq5evQoAuHbtGjIzMzVil0ql6NKli8XEbkiMycnJkMvlGtv4+/sjIiLCbK/j4MGDqFevHho3bow333wTWVlZ6sfMGW9OTg4AwMvLC4Dln9/y8apY6vlVKBTYvHkzCgoKEB0dbfHnt3y8KpZ4ft955x08//zzePbZZzXWm+scP3GDotfEvXv3oFAo4OPjo7Hex8cHmZmZZorqsfbt22P9+vVo3Lgx7ty5g9mzZ6Njx474999/1fHpiv3GjRvmCFeLITFmZmbC3t4enp6eWtuY4z3o1asXXn75ZQQHB+PatWv4+OOP0b17dyQnJ0MqlZotXsYYJk6ciKeffhoREREALPv86ooXsMzze+7cOURHR6O4uBguLi746aef0KxZM/WXsKWd34riBSzz/G7evBn//PMPTp48qfWYuT7DlCirofz0XIyxGk3ZZSy9evVS327RogWio6MRFhaGdevWqS/QW2rsZVUnRnO9jkGDBqlvR0REICoqCsHBwdi1axcGDBhQ4X6mjnfMmDE4e/Ys/vjjD63HLPH8VhSvJZ7fJk2a4PTp03j48CG2bduGYcOG4dChQ+rHLe38VhRvs2bNLO783rx5E+PHj8e+ffvg4OBQ4Xa1fY6p6rUKvL29IRaLtX6VZGVlaf3CsQTOzs5o0aIFLl26pG79asmxGxKjr68vZDIZHjx4UOE25uTn54fg4GBcunQJgHniHTt2LH755RccOHBAY0o5Sz2/FcWriyWcX3t7ezRs2BBRUVGYO3cuWrVqhS+//NJiz29F8epi7vObnJyMrKwsREZGws7ODnZ2djh06BD+97//wc7OTv2ctX2OKVFWgb29PSIjI5GUlKSxPikpCR07djRTVBUrKSnBhQsX4Ofnh9DQUPj6+mrELpPJcOjQIYuJ3ZAYIyMjIZFINLbJyMjA+fPnLeJ1ZGdn4+bNm/Dz8wNQu/EyxjBmzBhs374d+/fvR2hoqMbjlnZ+K4tXF3Oe34owxlBSUmJx57eyeHUx9/nt0aMHzp07h9OnT6uXqKgovP766zh9+jQaNGhgnnNcrSZATzBV95BVq1axlJQUNmHCBObs7MyuX79u7tDYpEmT2MGDB9nVq1fZ8ePHWe/evZmrq6s6tnnz5jF3d3e2fft2du7cOfbqq6/WeveQvLw8durUKXbq1CkGgC1cuJCdOnVK3b3GkBjj4+NZYGAg++2339g///zDunfvbrLm6vrizcvLY5MmTWLHjh1j165dYwcOHGDR0dEsICDALPG+/fbbzN3dnR08eFCjuX9hYaF6G0s6v5XFa2nnlzHGpkyZwg4fPsyuXbvGzp49y6ZOncpEIhHbt28fY8yyzm9l8Vri+dWlbKtXxsxzjilRVsOSJUtYcHAws7e3Z23atNFozm5Oqv5EEomE+fv7swEDBrB///1X/bhSqWTTp09nvr6+TCqVsmeeeYadO3euVmM8cOAAA6C1DBs2zOAYi4qK2JgxY5iXlxdzdHRkvXv3ZmlpabUeb2FhIYuNjWV169ZlEomE1a9fnw0bNkwrltqKV1ecANiaNWvU21jS+a0sXks7v4wxNmLECPX/ft26dVmPHj3USZIxyzq/lcVriedXl/KJ0hznmKbZIoQQQvSga5SEEEKIHpQoCSGEED0oURJCCCF6UKIkhBBC9KBESQghhOhBiZIQQgjRgxIlIYQQogclSkIIIUQPSpSEEIMJgoAdO3aYOwxCahUlSkKsxPDhwyEIgtbSs2dPc4dGiE2j+SgJsSI9e/bEmjVrNNZJpVIzRUPIk4FKlIRYEalUCl9fX41FNZO7IAhYtmwZevXqBUdHR4SGhmLr1q0a+587dw7du3eHo6Mj6tSpg7feegv5+fka26xevRrNmzeHVCqFn58fxowZo/H4vXv38MILL8DJyQmNGjXCL7/8YtoXTYiZUaIkxIZ8/PHHePHFF3HmzBkMHjwYr776Ki5cuAAAKCwsRM+ePeHp6YmTJ09i69at+O233zQS4bJly/DOO+/grbfewrlz5/DLL7+gYcOGGs8xY8YMDBw4EGfPnkVcXBxef/113L9/v1ZfJyG1qtrzjhBCatWwYcOYWCxmzs7OGsvMmTMZY3zaqvj4eI192rdvz95++23GGGMrVqxgnp6eLD8/X/34rl27mEgkYpmZmYwxxvz9/dm0adMqjAEA++ijj9T38/PzmSAI7P/+7/+M9joJsTR0jZIQK9KtWzcsW7ZMY52Xl5f6dnR0tMZj0dHROH36NADgwoULaNWqFZydndWPd+rUCUqlEqmpqRAEAbdv30aPHj30xtCyZUv1bWdnZ7i6uiIrK6u6L4kQi0eJkhAr4uzsrFUVWhlBEAAAjDH1bV3bODo6GnQ8iUSita9SqaxSTIRYE7pGSYgNOX78uNb9pk2bAgCaNWuG06dPo6CgQP340aNHIRKJ0LhxY7i6uiIkJAS///57rcZMiKWjEiUhVqSkpASZmZka6+zs7ODt7Q0A2Lp1K6KiovD0009j06ZNOHHiBFatWgUAeP311zF9+nQMGzYMiYmJuHv3LsaOHYshQ4bAx8cHAJCYmIj4+HjUq1cPvXr1Ql5eHo4ePYqxY8fW7gslxIJQoiTEiuzZswd+fn4a65o0aYKLFy8C4C1SN2/ejISEBPj6+mLTpk1o1qwZAMDJyQl79+7F+PHj0bZtWzg5OeHFF1/EwoUL1ccaNmwYiouLsWjRIrz33nvw9vbGSy+9VHsvkBALJDDGmLmDIITUnCAI+Omnn9C/f39zh0KITaFrlIQQQogelCgJIYQQPegaJSE2gq6iEGIaVKIkhBBC9KBESQghhOhBiZIQQgjRgxIlIYQQogclSkIIIUQPSpSEEEKIHpQoCSGEED0oURJCCCF6/D/TTXfpoGk2bAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 500x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# 创建图形\n",
    "plt.figure(figsize=(5, 3))\n",
    "\n",
    "plt.plot(x_axis_epoch, y_axis_train_error, label='Train Error', color='blue')\n",
    "plt.plot(x_axis_epoch, y_axis_test_error, label='Test Error', color='red')\n",
    "\n",
    "# 添加标题和标签\n",
    "plt.title('Train Error and Test Error')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Error')\n",
    "\n",
    "# 添加图例\n",
    "plt.grid(True)\n",
    "plt.legend()\n",
    "\n",
    "# 显示图形\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad6da4f1",
   "metadata": {},
   "source": [
    "# Batch Gradient Descent\n",
    "利用batch gradient descent 加速收敛"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c19f2457",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys, numpy as np\n",
    "from keras.datasets import mnist\n",
    "\n",
    "# load the MNIST dataset\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "\n",
    "# 训练数据一维化，归一化\n",
    "images, labels = (x_train[0:1000].reshape(1000,28*28) / 255, y_train[0:1000])\n",
    "# 训练数据one-hot编码\n",
    "one_hot_labels = np.zeros((len(labels),10))\n",
    "for i,label in enumerate(labels):\n",
    "    one_hot_labels[i][label] = 1\n",
    "labels = one_hot_labels\n",
    "\n",
    "# 测试数据归一化\n",
    "test_images = x_test.reshape(len(x_test),28*28) / 255  \n",
    "# 测试数据one-hot编码\n",
    "test_labels = np.zeros((len(y_test),10))\n",
    "for i,label in enumerate(y_test):\n",
    "    test_labels[i][label] = 1\n",
    "\n",
    "np.random.seed(1)\n",
    "\n",
    "def relu(x):\n",
    "    return (x >= 0) * x  # ReLU激活函数\n",
    "\n",
    "def relu2deriv(output):\n",
    "    return output>=0  # ReLU函数的导数\n",
    "\n",
    "batch_size = 100\n",
    "alpha, epochs, hidden_size, pixels_per_image, num_labels = (0.001, 400, 100, 784, 10)\n",
    "\n",
    "# 权重初始化\n",
    "weights_0_1 = 0.2*np.random.random((pixels_per_image,hidden_size)) - 0.1\n",
    "weights_1_2 = 0.2*np.random.random((hidden_size,num_labels)) - 0.1\n",
    "\n",
    "# 可视化数据记录\n",
    "x_axis_epoch = []\n",
    "y_axis_train_error = []\n",
    "y_axis_test_error = []\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ef39a336",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I:0 Train-Err:1.284 Train-Acc:0.165 Test-Err:0.815 Test-Acc:0.3832\n",
      "I:10 Train-Err:0.591 Train-Acc:0.672 Test-Err:0.568 Test-Acc:0.7173\n",
      "I:20 Train-Err:0.532 Train-Acc:0.729 Test-Err:0.510 Test-Acc:0.7571\n",
      "I:30 Train-Err:0.498 Train-Acc:0.754 Test-Err:0.485 Test-Acc:0.7793\n",
      "I:40 Train-Err:0.489 Train-Acc:0.749 Test-Err:0.468 Test-Acc:0.7877\n",
      "I:50 Train-Err:0.468 Train-Acc:0.775 Test-Err:0.458 Test-Acc:0.793\n",
      "I:60 Train-Err:0.452 Train-Acc:0.799 Test-Err:0.452 Test-Acc:0.7995\n",
      "I:70 Train-Err:0.453 Train-Acc:0.792 Test-Err:0.446 Test-Acc:0.803\n",
      "I:80 Train-Err:0.457 Train-Acc:0.786 Test-Err:0.451 Test-Acc:0.7968\n",
      "I:90 Train-Err:0.454 Train-Acc:0.799 Test-Err:0.447 Test-Acc:0.795\n",
      "I:100 Train-Err:0.447 Train-Acc:0.796 Test-Err:0.448 Test-Acc:0.793\n",
      "I:110 Train-Err:0.426 Train-Acc:0.816 Test-Err:0.441 Test-Acc:0.7943\n",
      "I:120 Train-Err:0.431 Train-Acc:0.813 Test-Err:0.442 Test-Acc:0.7966\n",
      "I:130 Train-Err:0.434 Train-Acc:0.816 Test-Err:0.441 Test-Acc:0.7906\n",
      "I:140 Train-Err:0.437 Train-Acc:0.822 Test-Err:0.447 Test-Acc:0.7874\n",
      "I:150 Train-Err:0.414 Train-Acc:0.823 Test-Err:0.443 Test-Acc:0.7899\n",
      "I:160 Train-Err:0.427 Train-Acc:0.811 Test-Err:0.438 Test-Acc:0.797\n",
      "I:170 Train-Err:0.418 Train-Acc:0.828 Test-Err:0.440 Test-Acc:0.7884\n",
      "I:180 Train-Err:0.407 Train-Acc:0.834 Test-Err:0.436 Test-Acc:0.7935\n",
      "I:190 Train-Err:0.410 Train-Acc:0.831 Test-Err:0.434 Test-Acc:0.7935\n",
      "I:200 Train-Err:0.416 Train-Acc:0.829 Test-Err:0.435 Test-Acc:0.7972\n",
      "I:210 Train-Err:0.409 Train-Acc:0.83 Test-Err:0.434 Test-Acc:0.7923\n",
      "I:220 Train-Err:0.396 Train-Acc:0.832 Test-Err:0.433 Test-Acc:0.8032\n",
      "I:230 Train-Err:0.393 Train-Acc:0.853 Test-Err:0.431 Test-Acc:0.8036\n",
      "I:240 Train-Err:0.397 Train-Acc:0.844 Test-Err:0.430 Test-Acc:0.8047\n",
      "I:250 Train-Err:0.386 Train-Acc:0.843 Test-Err:0.429 Test-Acc:0.8028\n",
      "I:260 Train-Err:0.394 Train-Acc:0.843 Test-Err:0.431 Test-Acc:0.8038\n",
      "I:270 Train-Err:0.384 Train-Acc:0.845 Test-Err:0.428 Test-Acc:0.8014\n",
      "I:280 Train-Err:0.401 Train-Acc:0.846 Test-Err:0.430 Test-Acc:0.8067\n",
      "I:290 Train-Err:0.383 Train-Acc:0.851 Test-Err:0.428 Test-Acc:0.7975\n",
      "I:300 Train-Err:0.391 Train-Acc:0.847 Test-Err:0.430 Test-Acc:0.8052\n",
      "I:310 Train-Err:0.387 Train-Acc:0.852 Test-Err:0.429 Test-Acc:0.7977\n",
      "I:320 Train-Err:0.389 Train-Acc:0.845 Test-Err:0.429 Test-Acc:0.8035\n",
      "I:330 Train-Err:0.393 Train-Acc:0.853 Test-Err:0.429 Test-Acc:0.7982\n",
      "I:340 Train-Err:0.395 Train-Acc:0.841 Test-Err:0.430 Test-Acc:0.7953\n",
      "I:350 Train-Err:0.389 Train-Acc:0.846 Test-Err:0.434 Test-Acc:0.7981\n",
      "I:360 Train-Err:0.389 Train-Acc:0.851 Test-Err:0.429 Test-Acc:0.7974\n",
      "I:370 Train-Err:0.385 Train-Acc:0.848 Test-Err:0.432 Test-Acc:0.7969\n",
      "I:380 Train-Err:0.386 Train-Acc:0.835 Test-Err:0.431 Test-Acc:0.7991\n",
      "I:390 Train-Err:0.369 Train-Acc:0.845 Test-Err:0.428 Test-Acc:0.8071\n",
      "I:399 Train-Err:0.376 Train-Acc:0.843 Test-Err:0.429 Test-Acc:0.7969\n"
     ]
    }
   ],
   "source": [
    "\n",
    "for j in range(epochs):\n",
    "    error, correct_cnt = (0.0, 0)  # 训练集误差，预测正确个数计数\n",
    "    \n",
    "    weight_update_num = int(len(images) / batch_size) # 每轮迭代更新权重次数\n",
    "    for i in range(weight_update_num):  # BGD\n",
    "        batch_start, batch_end = ((i * batch_size),((i+1)*batch_size))\n",
    "\n",
    "        # 前向传播\n",
    "        layer_0 = images[batch_start:batch_end]\n",
    "        layer_1 = relu(np.dot(layer_0,weights_0_1))\n",
    "        dropout_mask = np.random.randint(2, size=layer_1.shape) # 模拟dropout\n",
    "        layer_1 *= dropout_mask * 2                             # *2是为了保证在训练时保持神经元输出的期望不变\n",
    "        layer_2 = np.dot(layer_1,weights_1_2)\n",
    "\n",
    "        error += np.sum((labels[batch_start:batch_end] - layer_2) ** 2)  # 均方差\n",
    "        for k in range(batch_size):\n",
    "            correct_cnt += int(np.argmax(layer_2[k:k+1]) == \\\n",
    "                               np.argmax(labels[batch_start+k:batch_start+k+1])) # 预测正确个数计数\n",
    "\n",
    "            # 反向传播\n",
    "            layer_2_delta = (labels[batch_start:batch_end]-layer_2)/batch_size\n",
    "            layer_1_delta = layer_2_delta.dot(weights_1_2.T)* relu2deriv(layer_1)\n",
    "            layer_1_delta *= dropout_mask\n",
    "        \n",
    "            # 更新权重\n",
    "            weights_1_2 += alpha * layer_1.T.dot(layer_2_delta)\n",
    "            weights_0_1 += alpha * layer_0.T.dot(layer_1_delta)\n",
    "\n",
    "    # 每隔10次迭代输出结果，并记录测试集准确率\n",
    "    if(j % 10 == 0 or j == epochs-1):\n",
    "        \n",
    "        # 测试集\n",
    "        test_error, test_correct_cnt = (0.0, 0)   # 测试集误差，预测正确个数计数\n",
    "\n",
    "        for i in range(len(test_images)):\n",
    "\n",
    "            layer_0 = test_images[i:i+1]\n",
    "            layer_1 = relu(np.dot(layer_0,weights_0_1))\n",
    "            layer_2 = np.dot(layer_1, weights_1_2)\n",
    "            \n",
    "            test_error += np.sum((test_labels[i:i+1] - layer_2) ** 2)\n",
    "            test_correct_cnt += int(np.argmax(layer_2) == \\\n",
    "                                            np.argmax(test_labels[i:i+1]))\n",
    "        \n",
    "        sys.stdout.write(\"I:\"+str(j)+ \\\n",
    "                     \" Train-Err:\" + str(error/float(len(images)))[0:5] +\\\n",
    "                     \" Train-Acc:\" + str(correct_cnt/float(len(images))))\n",
    "        sys.stdout.write(\" Test-Err:\" + str(test_error/float(len(test_images)))[0:5] +\\\n",
    "                         \" Test-Acc:\" + str(test_correct_cnt/float(len(test_images))))\n",
    "        print()  # 换行\n",
    "\n",
    "        x_axis_epoch.append(j) # 记录轮次\n",
    "        y_axis_train_error.append(error/float(len(images))) # 记录训练集误差\n",
    "        y_axis_test_error.append(test_error/float(len(test_images))) # 记录测试集误差\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "7a59c3d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcoAAAE6CAYAAACWDhLFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABPL0lEQVR4nO3deVxUVf8H8M+dhWHfRFkEETcQTU3RXDI1BcU028ynTU1bjNTKNs1yy7KnfMx6Snsql7ZfmmlqSSpluGSWqZQLoiWKCkjiwg4Dc35/XGdgGBhgGJjFz/v1ui9mztx75zt3hvnOOfeccyUhhAARERHVSGHrAIiIiOwZEyUREZEZTJRERERmMFESERGZwURJRERkBhMlERGRGUyUREREZjBREhERmcFESUREZAYTJTUpSZLqtSQnJzfqeebNmwdJkqwT9DXm4p04caJVn8sR1XXMk5OT6/3+W8OxY8cwb948nD59ul7rr169ukk/k+Q8VLYOgJzbL7/8YnT/1VdfxU8//YQdO3YYlUdHRzfqeR555BGMGDGiUfuoyT333INnn33WpLxly5ZWfy5n07NnT5P3/84770T79u2xePFiqz/fsWPHMH/+fAwePBht27at93arVq1CVFSUSXljP5PkPJgoqUn17dvX6H7Lli2hUChMyqsrKiqCu7t7vZ8nNDQUoaGhFsVoTmBgYJ2x1qS2+IUQKCkpgZubm8UxFRcXw9XV1eo1aGvz9vY2OXYajQa+vr4WHdOm0rVrV8TExDRoG3PvozXen4Z+/qlpsemVbG7w4MHo2rUrdu3ahf79+8Pd3R2TJk0CAKxduxZxcXEIDg6Gm5sbOnfujJkzZ6KwsNBoHzU1A7Zt2xajRo3C1q1b0bNnT7i5uSEqKgorV660avwTJ06Ep6cnDh8+jLi4OHh5eWHo0KEA5ObbqVOn4oMPPkDnzp2h0WjwySefAAD27NmDoUOHwsvLC+7u7ujfvz+2bNlitG998+D27dsxadIktGzZEu7u7igtLa0xlpKSEjz77LPo0aMHfHx84O/vj379+mHTpk0m6+pj++yzz9C5c2e4u7uje/fu+O6770zW3bJlC3r06AGNRoOIiAir1gizs7Px+OOPIzQ0FC4uLoiIiMD8+fNRXl5utN7y5cvRvXt3eHp6wsvLC1FRUXjppZcAyMdp7NixAIAhQ4YYmk9Xr15tlRhrex/NvT86nQ5vvvkmoqKioNFo0KpVK4wfPx7nzp0z2re5zz/ZB9YoyS5kZWXhwQcfxAsvvIDXX38dCoX8G+7kyZMYOXIknn76aXh4eOD48eP497//jd9++82k+bYmf/zxB5599lnMnDkTgYGB+PjjjzF58mR06NABt9xyS53bCyFMvrABQKlUGiXmsrIy3H777Xj88ccxc+ZMo202btyI3bt3Y86cOQgKCkKrVq2wc+dOxMbGolu3blixYgU0Gg2WLVuG0aNH48svv8S4ceOMnm/SpEm47bbb8Nlnn6GwsBBqtbrGeEtLS3Hp0iU899xzaN26NcrKyvDDDz/grrvuwqpVqzB+/Hij9bds2YL9+/djwYIF8PT0xJtvvok777wTaWlpaNeuHQDgxx9/xJgxY9CvXz+sWbMGFRUVePPNN3HhwoU6j19dsrOz0adPHygUCsyZMwft27fHL7/8goULF+L06dNYtWoVAGDNmjVISEjAtGnTsHjxYigUCvz11184duwYAOC2227D66+/jpdeegnvv/8+evbsCQBo3759nTFUVFSYvMeSJEGpVBqV1fQ+7t+/H0DN788TTzyBDz/8EFOnTsWoUaNw+vRpvPLKK0hOTsbBgwcREBBg2Hdtn3+yE4KoGU2YMEF4eHgYlQ0aNEgAED/++KPZbXU6ndBqtWLnzp0CgPjjjz8Mj82dO1dU/ziHh4cLV1dXcebMGUNZcXGx8Pf3F48//nidsQKodfnss8+MXhMAsXLlyhr34ePjIy5dumRU3rdvX9GqVSuRn59vKCsvLxddu3YVoaGhQqfTCSGEWLVqlQAgxo8fX2e8NSkvLxdarVZMnjxZ3HjjjSaxBQYGiry8PENZdna2UCgUYtGiRYaym266SYSEhIji4mJDWV5envD39zc55nUJDw8Xt912m+H+448/Ljw9PY3eIyGEWLx4sQAgjh49KoQQYurUqcLX19fsvtetWycAiJ9++qleseiPbU2LUqk0Wre297G29yc1NVUAEAkJCUblv/76qwAgXnrpJUNZfT//ZDv82UJ2wc/PD7feeqtJ+alTp3D//fcjKCgISqUSarUagwYNAgCkpqbWud8ePXqgTZs2hvuurq7o1KkTzpw5U6+47r33Xuzfv99kGTlypMm6d999d437uPXWW+Hn52e4X1hYiF9//RX33HMPPD09DeVKpRIPPfQQzp07h7S0tHrtuybr1q3DgAED4OnpCZVKBbVajRUrVtR4vIYMGQIvLy/D/cDAQLRq1cpwfAoLC7F//37cddddcHV1Nazn5eWF0aNH1zum2nz33XcYMmQIQkJCUF5eblji4+MBADt37gQA9OnTB1euXMF9992HTZs24eLFi41+br1PP/3U5P399ddfTdar/j5WVf39+emnnwDApHd0nz590LlzZ/z4449G5bV9/sk+sOmV7EJwcLBJWUFBAQYOHAhXV1csXLgQnTp1gru7O86ePYu77roLxcXFde63RYsWJmUajaZe2wJy56P6dPRwd3eHt7d3jY9Vf22XL1+GEKLG1xwSEgIAyM3NNbuP2mzYsAH33nsvxo4di+effx5BQUFQqVRYvnx5jedm6zo+ly9fhk6nQ1BQkMl6NZU11IULF/Dtt9/W2pSsT4gPPfQQysvL8dFHH+Huu++GTqdD7969sXDhQsTGxjYqhs6dO9frPTb3HlR/TP/+1fYeV/+hVt/3l2yDiZLsQk09BHfs2IHMzEwkJycbapEAcOXKlWaMrH7M9XCs/pifnx8UCgWysrJM1s3MzAQAo/NXde2/qs8//xwRERFYu3at0Ta1df6pi5+fHyRJQnZ2tsljNZU1VEBAALp164bXXnutxsf1PxwA4OGHH8bDDz+MwsJC7Nq1C3PnzsWoUaNw4sQJhIeHNzqWujTkPdb/AMnKyjLpjZ2ZmWnx+0u2waZXslv6Lw+NRmNU/r///c8W4ViNh4cHbrrpJmzYsMGoZqvT6fD5558jNDQUnTp1smjfkiTBxcXF6Is3Ozu7xl6v9Y21T58+2LBhA0pKSgzl+fn5+Pbbby3aZ1WjRo3CkSNH0L59e8TExJgsVRNl1Zji4+Mxe/ZslJWV4ejRowAqPyf1bS1oSvpm1M8//9yofP/+/UhNTTX0iibHwBol2a3+/fvDz88PU6ZMwdy5c6FWq/HFF1/gjz/+aLYYLly4gH379pmUe3t7N2pA+qJFixAbG4shQ4bgueeeg4uLC5YtW4YjR47gyy+/tLiGMWrUKGzYsAEJCQm45557cPbsWbz66qsIDg7GyZMnLdrnq6++ihEjRiA2NhbPPvssKioq8O9//xseHh64dOmSRfvUW7BgAZKSktC/f39Mnz4dkZGRKCkpwenTp5GYmIgPPvgAoaGhePTRR+Hm5oYBAwYgODgY2dnZWLRoEXx8fNC7d28A8nhIAPjwww/h5eUFV1dXRERE1Ni8XNWRI0dq7Nncvn17iyeWiIyMxGOPPYb//ve/UCgUiI+PN/R6DQsLwzPPPGPRfsk2mCjJbrVo0QJbtmzBs88+iwcffBAeHh4YM2YM1q5da+j+39S+/vprfP311yblAwYMwJ49eyze76BBg7Bjxw7MnTsXEydOhE6nQ/fu3bF582aMGjXK4v0+/PDDyMnJwQcffICVK1eiXbt2mDlzJs6dO4f58+dbtM/Y2Fhs3LgRL7/8MsaNG4egoCAkJCSguLjY4n3qBQcH4/fff8err76Kt956C+fOnYOXlxciIiIwYsQIQ+eZgQMHYvXq1fjqq69w+fJlBAQE4Oabb8ann35qSGYRERFYunQp3nnnHQwePBgVFRVYtWpVndMNPvzwwzWWf/TRR3jkkUcsfm3Lly9H+/btsWLFCrz//vvw8fHBiBEjsGjRojqTN9kXSQghbB0EERGRveI5SiIiIjOYKImIiMxgoiQiIjKDiZKIiMgMJkoiIiIzmCiJiIjMuO7GUep0OmRmZsLLy4vTRhERXceEEMjPz0dISIjZS5tdd4kyMzMTYWFhtg6DiIjsxNmzZ03m5K3qukuU+ksKnT17ttarPdSHVqvF9u3bERcXV+uVD+wJ421ajhYv4HgxM96mdT3Gm5eXh7CwMKNLzdXkukuU+uZWb2/vRidK/aWVHOVDxXibjqPFCzhezIy3aV3P8dZ1Go6deYiIiMxgoiQiIjKDiZKIiMiM6+4cJRFRVUIIlJeXo6Kiwqr71Wq1UKlUKCkpsfq+m4IzxqtUKqFSqRo9FJCJkoiuW2VlZcjKykJRUZHV9y2EQFBQEM6ePesQY7adNV53d3cEBwfDxcXF4udioiSi65JOp0N6ejqUSiVCQkLg4uJi1QSh0+lQUFAAT09Ps4PZ7YWzxSuEQFlZGf755x+kp6ejY8eOFr8uJkoL5OcDgwcrkZ09FCdOAA7Qk5qIqikrK4NOp0NYWBjc3d2tvn+dToeysjK4uro6TOJxtnjd3NygVqtx5swZw7qWYKK0gKsrcPCgAoAn8vO18PCwdUREZClHSApkOWu8v/yEWECtBtzdBQAgL8/GwRARUZNiorSQflKfq1dtGwcRETUtJkoL6acGzM+3/95hRETmDB48GM8884ytw7BbTJQW8vZm0ysRNS9JkswuEydOtGi/GzZswIIFCxoV28SJE2uMacSIEY3arz1gZx4L6ZtemSiJqLlkZWUZbq9duxZz5sxBWlqaoczNzc1ofa1WW68Jw/39/aHT6ZDXyC+0ESNGYNWqVUZlGo2m1vVriq++MddnX9bCGqWF9ImSTa9EzkMIoLCw+Rch6hdfUFCQYfHx8YEkSYb7JSUl8PX1xVdffYXBgwfD1dUVn3/+OXJzc3HfffchNDQU7u7uuOGGG/Dll18a7bd602vbtm3x+uuvY9KkSfDy8kKbNm3w4Ycf1hmfRqMxijEoKAh+fn6GxyVJwgcffIAxY8bAw8MDCxcuxLx589CjRw+sXLkS7dq1g0ajgRACGRkZGDNmDDw9PeHt7Y17770XFy5cMOxr/vz5GDhwoMl2TYGJ0kLszEPkfIqKAE9P6yze3gqEhvrC21tR57rWnBjoxRdfxPTp05Gamorhw4ejpKQEvXr1wnfffYcjR47gsccew0MPPYRff/3V7H7+85//ICYmBocOHUJCQgKeeOIJHD9+vNHxzZ07F2PGjMHhw4cxadIkAMBff/2Fr776CuvXr0dKSgoA4I477sClS5ewc+dOJCUl4e+//8a4ceOM9pWeno5169YZbdcU2PRqIZ6jJCJ79PTTT+Ouu+4yKnvuuecMt6dNm4atW7di3bp1uOmmm2rdz8iRI5GQkABATr5vv/02kpOTERUVVes23333HTw9PY3KXnzxRbzyyiuG+/fff78hQeqVlZXhs88+Q8uWLQEASUlJ+PPPP5Geno6wsDAAwGeffYYuXbpg//796N27t2G7Tz/9FIGBgbXGZA1MlBaq7PVq2ziIyHrc3YGCAuvsS3/Oz9vbu85B79acGCgmJsbofkVFBd544w2sXbsW58+fR2lpKUpLS+FRx0wp3bp1M9zWN/Hm5OSY3WbIkCFYvny5UZm/v7/Z+AAgPDzckCQBIDU1FWFhYYYkCQDR0dHw9fVFamqqIVGGhYUZbddUmCgtVNmZh+coiZyFJMFqM23pdEBFhby/5pz8p3oC/M9//oO3334bS5cuxQ033AAPDw88/fTTKCsrM7uf6h1jJEmCTqer87k7dOjQoPhqKhNC1DjvbvXypph6sCZMlBby8ZH/sumViOzZ7t27MWbMGDz44IMA5JruyZMn0blzZxtHVrvo6GhkZGTg7NmzhlrlsWPHcPXqVZvEzURpIS8vnqMkIvvXoUMHrF+/Hnv37oWfnx+WLFmC7OzsJkk4paWlyM7ONipTqVQICAho0H6GDRuGbt264YEHHsDSpUtRXl6OhIQEDBo0qMam26bGXq8W4jhKInIEr7zyCnr27Inhw4dj8ODBCAoKwh133NEkz7V161YEBwcbLTfffHOD9yNJEjZu3Ag/Pz/ccsstGDZsGNq1a4e1a9c2QdR1Y43SQjxHSUS2NHHiRKOZeNq2bVvjOEJ/f39s3LjR7L6Sk5ONJhw4ffq0yTp1Db9YvXo1Vq9ebXadmuKbN28e5s2bZ1Lepk0bbNq0qdZ9zZ07t9mm3WON0kL6plf2eiUicm42TZS7du3C6NGjERISYqhqm7NhwwbExsaiZcuW8Pb2Rr9+/bBt27bmCbYafWceTjhAROTcbJooCwsL0b17d7z33nv1Wn/Xrl2IjY1FYmIiDhw4gCFDhmD06NE4dOhQE0dqSt/0WlIioY5e1kRE5MBseo4yPj4e8fHx9V5/6dKlRvdff/11bNq0Cd9++y1uvPFGK0dnnn7CAUBufm3RolmfnoiImolDd+bR6XTIz883mfmhKv0sFHr6k9VarRZarbYRz66Fi4uEsjIVcnO1hhqmvdK/1sa95ubDeJueo8Vs7Xi1Wi2EENDpdHUOpLeEvuOK/jnsnbPGq9PpIISAVquFUqk0eqy+nyWHTpT/+c9/UFhYiHvvvbfWdRYtWoT58+eblG/fvr3Rszp4eAxHWZkKiYl7EBHhGONEkpKSbB1CgzDepudoMVsrXpVKhaCgIBQUFNQ5S01j5DtYjz9ni7esrAzFxcXYtWsXysvLjR4rquds9JJoquuSNJAkSfjmm2/qPb7nyy+/xCOPPIJNmzZh2LBhta5XU40yLCwMFy9ehHcjqoFarRYdO+qQmemJH38sx8CBdnEYa6XVapGUlITY2Ngmu2abNTHepudoMVs73pKSEpw9exZt27aFq6urFSI0JoRAfn4+vLy8apyOzd44a7wlJSU4ffo0wsLCTN7nvLw8BAQE4OrVq2bzgUPWKNeuXYvJkydj3bp1ZpMkIF8fraYLh6rV6kb/s7m7y7MnFxWp4ADfMwCs87qbE+Nteo4Ws7XiraiogCRJUCgUdU5abgl9c6D+Oeyds8arUCggSVKNn5v6fo7s/2hU8+WXX2LixIn4v//7P9x22202jcXNTa7Gc3YeIiLnZdMaZUFBAf766y/D/fT0dKSkpMDf3x9t2rTBrFmzcP78eXz66acA5CQ5fvx4vPPOO+jbt69hTkE3Nzf46Ac2NiN3dyZKIiJnZ9Ma5e+//44bb7zRMLRjxowZuPHGGzFnzhwAQFZWFjIyMgzr/+9//0N5eTmefPJJo7kEn3rqKZvE7+4u95hioiSi5iBJktml6pR2DdWuXTuTa0nWpG3btjU+9xtvvGHxc9s7m9YoBw8eXOPcf3rV5w1MTk5u2oAaSF+j5Ow8RNQcsrKyDLfXrl2LOXPmIC0tzVDm5ubWLHEsWLAAjz76qFGZV9XB5VUIIVBRUQGVyjjdlJWVwcXFpcHPbel2jeFw5yjtCWuURE5GCKCwsPmXeg4+CAoKMiw+Pj6QJMmobNeuXejVqxdcXV3Rrl07zJ8/32hIxLx589CmTRtoNBqEhIRg+vTpAORKy5kzZ/DSSy9BqVTW2evVy8vL6HmDgoIMF19OTk6GJEnYtm0bYmJioNFosHv3bgwePBhTp07FjBkzEBAQgNjYWADAzp070adPH2g0GgQHB2PmzJlGMde2XXNyyF6v9oKdeYicTFER4OlplV0pAPjWd+WCAuBaorHUtm3b8OCDD+Ldd9/FwIED8ffff+Oxxx4DIF9p4+uvv8bbb7+NNWvWoEuXLsjOzsYff/wBQJ5Hu3v37hg/fjyefPJJq/R6feGFF7B48WK0a9cOvr6+AIBPPvkETzzxBH7++WcIIXD+/HmMHDkSEydOxKefforjx4/j0Ucfhaurq9EVRapv19yYKBuBnXmIyF689tprmDlzJiZMmABAPuf46quv4oUXXsDcuXORkZGBoKAgDBs2DGq1Gm3atEGfPn0AyJfiUiqV8PT0RFBQUJ2J8sUXX8TLL79sVPbdd99h8ODBhvsLFiwwqf116NABb775puH+7NmzERYWhvfeew+SJCEqKgqZmZl48cUXMWfOHEMc1bdrbkyUjcCmVyIn4+4u1+6sQH99R29v77praI2cJQwADhw4gP379+O1114zlFVUVKCkpARFRUUYO3Ysli5dinbt2mHEiBEYOXIkRo8ebXLusD6ef/55k45DrVu3NrofExNjsl31stTUVPTr18+oqXfAgAEoKCjAuXPn0KZNm1r31ZyYKBuBnXmInIwkNboJ1ECnAyoq5P01wwB+nU6H+fPn46677jJ5zNXVFWFhYUhLS0NSUhJ++OEHJCQk4K233sLOnTsbPIFDQEAAOnToYHYdjxqOY/UyIYTJ+VB902rV8pr21ZyYKBvBzY01SiKyDz179kRaWprZBObm5obbb78dt99+O5588klERUXh8OHD6NmzJ1xcXFBRUdGMEQPR0dFYv369UcLcu3cvvLy8TGqotsRE2Qg8R0lE9mLOnDkYNWoUwsLCMHbsWCgUCvz55584fPgwFi5ciNWrV6OiogI33XQT3N3d8dlnn8HNzQ3h4eEAgPDwcOzduxfnz5+Hm5sbAgICan2u/Px8w4Qveu7u7g2ePzshIQFLly7FtGnTMHXqVKSlpWHu3LmYMWOGXU2jZz+ROCCeoyQiezF8+HB89913SEpKQu/evdG3b18sWbLEkAh9fX3x0UcfYcCAAejWrRt+/PFHfPvtt2hx7WK68+fPR0ZGBjp27IiWLVuafa45c+YYTfoSHByMF154ocExt27dGomJifjtt9/QvXt3TJkyBZMnTzbpKGRrrFE2gr5GWVwMaLVwmInRicjxTZw40aRDzfDhwzF8+PAa17/jjjvMXp2pb9++2LNnT52dj06fPm02rtomkqltwphBgwbht99+q3V/9jDRDGuUjaAfRwmwVklE5KyYKBtBpRJwc5N/OTFREhE5JybKRtKfu2aiJCJyTkyUjcRESUTk3JgoG8nbW2565aQDRI7JFnOHUvOxxvvLRNlIrFESOSb9bDRFRUU2joSakv79bejsQ1VxeEgj6S/BxkRJ5FiUSiV8fX2Rk5MDQB4wX9flpRpCp9OhrKwMJSUldjV4vjbOFq8QAkVFRcjJyYGvry+USqXFz8VE2UisURI5rqCgIAAwJEtrEkKguLgYbm5uVk3ATcVZ4/X19TW8z5ZiomwkHx8ODyFyVJIkITg4GK1atYJWq7XqvrVaLXbt2oVbbrmlUc1+zcUZ41Wr1Y2qSeoxUTaSvumVnXmIHJdSqbTKF2r1fZaXl8PV1dUhEg/jrZ39N0TbOTa9EhE5NybKRmKiJCJybkyUjeTlxXOURETOjImykXx85L9MlEREzomJspH0Ta/szENE5JyYKBuJTa9ERM6NibKR2JmHiMi5MVE2kv4cZXExYOXxykREZAeYKBtJP+EAwFolEZEzYqJsJLUacHOTbzNREhE5HyZKK+B5SiIi58VEaQVMlEREzouJ0go46QARkfNiorQCTjpAROS8mCitgE2vRETOi4nSCpgoiYicl00T5a5duzB69GiEhIRAkiRs3Lixzm127tyJXr16wdXVFe3atcMHH3zQ9IHWgYmSiMh52TRRFhYWonv37njvvffqtX56ejpGjhyJgQMH4tChQ3jppZcwffp0rF+/vokjNY+deYiInJfKlk8eHx+P+Pj4eq//wQcfoE2bNli6dCkAoHPnzvj999+xePFi3H333U0UZd3YmYeIyHnZNFE21C+//IK4uDijsuHDh2PFihXQarVQq9Um25SWlqK0tNRwP+9atU+r1ULbiMlZ9dtqtVp4eCgAKHHlig5abYXF+2xKVeN1BIy36TlazIy3aV2P8dZ3W4dKlNnZ2QgMDDQqCwwMRHl5OS5evIjg4GCTbRYtWoT58+eblG/fvh3u7u6NjikpKQl//90aQAzS03ORmLi30ftsSklJSbYOoUEYb9NztJgZb9O6nuItKiqq13oOlSgBQJIko/tCiBrL9WbNmoUZM2YY7ufl5SEsLAxxcXHw1reZWkCr1SIpKQmxsbEQwgVvvw24uLTAyJEjLd5nU6oab001b3vDeJueo8XMeJvW9RhvXj07ljhUogwKCkJ2drZRWU5ODlQqFVq0aFHjNhqNBhqNxqRcrVZb5cOgVqvRooV8GPPyFFCr7XvEjbVed3NhvE3P0WJmvE3reoq3vtvZ97d6Nf369TOpZm/fvh0xMTE2fWM5PISIyHnZNFEWFBQgJSUFKSkpAOThHykpKcjIyAAgN5uOHz/esP6UKVNw5swZzJgxA6mpqVi5ciVWrFiB5557zhbhGzBREhE5L5s2vf7+++8YMmSI4b7+XOKECROwevVqZGVlGZImAERERCAxMRHPPPMM3n//fYSEhODdd9+16dAQoDJRFhcDWq18jUoiInIONk2UgwcPNnTGqcnq1atNygYNGoSDBw82YVQNV7VPUH4+4O9vu1iIiMi6HOocpb1SqwE3N/k2Jx0gInIuTJRWwvOURETOiYnSSpgoiYicExOllTBREhE5JyZKK+EVRIiInBMTpZXwCiJERM6JidJK2PRKROScmCithImSiMg5MVFaCRMlEZFzYqK0EnbmISJyTkyUVsLOPEREzomJ0krY9EpE5JyYKK2EiZKIyDkxUVoJz1ESETknJkor4TlKIiLnxERpJWx6JSJyTkyUVqJPlMXFgFZr21iIiMh6mCitRJ8oASA/33ZxEBGRdTFRWolaDbi5ybfZ/EpE5DwanCjLy8uhUqlw5MiRpojHobFDDxGR82lwolSpVAgPD0dFRUVTxOPQ2KGHiMj5WNT0+vLLL2PWrFm4dOmSteNxaEyURETOR2XJRu+++y7++usvhISEIDw8HB4eHkaPHzx40CrBORomSiIi52NRorzjjjusHIZz4Ow8RETOx6JEOXfuXGvH4RTYmYeIyPlYlCj1Dhw4gNTUVEiShOjoaNx4443WisshsemViMj5WJQoc3Jy8K9//QvJycnw9fWFEAJXr17FkCFDsGbNGrRs2dLacToEJkoiIudjUa/XadOmIS8vD0ePHsWlS5dw+fJlHDlyBHl5eZg+fbq1Y3QYTJRERM7Hohrl1q1b8cMPP6Bz586GsujoaLz//vuIi4uzWnCOhp15iIicj0U1Sp1OB7VabVKuVquh0+kaHZSjYmceIiLnY1GivPXWW/HUU08hMzPTUHb+/Hk888wzGDp0qNWCczRseiUicj4WJcr33nsP+fn5aNu2Ldq3b48OHTogIiIC+fn5+O9//2vtGB0GEyURkfOx6BxlWFgYDh48iKSkJBw/fhxCCERHR2PYsGHWjs+h8BwlEZHzaXCiLC8vh6urK1JSUhAbG4vY2NimiMv+VVTA9Z9/jIp4jpKIyPnw6iGWyMqCytcXsU88AWi1hmJ9oiwuNiomIiIHZvOrhyxbtgwRERFwdXVFr169sHv3brPrf/HFF+jevTvc3d0RHByMhx9+GLm5uY2Oo0ECAwGlEoryciA93VDs5VW5Sn5+84ZERERNw6JE+e6772L37t0ICQlBZGQkevbsabTU19q1a/H0009j9uzZOHToEAYOHIj4+HhkZGTUuP6ePXswfvx4TJ48GUePHsW6deuwf/9+PPLII5a8DMspFECnTgAAKS3NUOziAri6yrd5npKIyDnY9OohS5YsweTJkw2JbunSpdi2bRuWL1+ORYsWmay/b98+tG3b1jD7T0REBB5//HG8+eabtT5HaWkpSktLDffzrmUwrVYLbSPaR6WOHaFKSYHu2DFoR482lPv4qFBSIiE3V4vWrS3evdXpX2tjXnNzYrxNz9FiZrxN63qMt77bWtSZBwAmTZqEsLCwhm5uUFZWhgMHDmDmzJlG5XFxcdi7d2+N2/Tv3x+zZ89GYmIi4uPjkZOTg6+//hq33XZbrc+zaNEizJ8/36R8+/btcHd3tzj+TioVOgPISk5Gyg03GMqVyqEAPLFt2z6cO2d/F7ZOSkqydQgNwnibnqPFzHib1vUUb1FRUb3Wa3CiVKlUWLx4MSZMmNDgoKq6ePEiKioqEBgYaFQeGBiI7OzsGrfp378/vvjiC4wbNw4lJSUoLy/H7bffbnbs5qxZszBjxgzD/by8PISFhSEuLg7e+t43FtDl5QFffonQggKEjBxpKA8KUiIzE+jSpR9GjhQW79/atFotkpKSEBsbW+OsSvaG8TY9R4uZ8Tat6zHevHqeI7Oo6XXo0KFITk7GxIkTLdnciCRJRveFECZleseOHcP06dMxZ84cDB8+HFlZWXj++ecxZcoUrFixosZtNBoNNBqNSblarW7Uh0EbHQ0AUJw4AaVKBVyLWT+WsqhIBXv8rDX2dTc3xtv0HC1mxtu0rqd467udRYkyPj4es2bNwpEjR9CrVy94eHgYPX777bfXuY+AgAAolUqT2mNOTo5JLVNv0aJFGDBgAJ5//nkAQLdu3eDh4YGBAwdi4cKFCA4OtuTlWKZjRwhJgnT5MvDPP0CrVgA4Ow8RkbOxKFE+8cQTAOTOONVJklSvMZYuLi7o1asXkpKScOeddxrKk5KSMGbMmBq3KSoqgkplHLJSqQQg10SblZsbilq1gseFC8Dx44ZEydl5iIici8VXD6ltachEBDNmzMDHH3+MlStXIjU1Fc888wwyMjIwZcoUAPL5xfHjxxvWHz16NDZs2IDly5fj1KlT+PnnnzF9+nT06dMHISEhlryURinQd2s9ftxQxtl5iIicS4MS5ciRI3G1SgZ47bXXcOXKFcP93NxcRF87d1cf48aNw9KlS7FgwQL06NEDu3btQmJiIsLDwwEAWVlZRmMqJ06ciCVLluC9995D165dMXbsWERGRmLDhg0NeRlWkx8aKt+oIVGyRklE5Bwa1PS6bds2ozGJ//73v3HffffB19cXgDx0JK3KAPz6SEhIQEJCQo2PrV692qRs2rRpmDZtWoOeo6kYapRVXjMTJRGRc2lQjbL6ecBmPy9oZwpYoyQicnoWnaMkWb6+RpmeDpSUAGBnHiIiZ9OgRClJkskYx9rGPF4Pynx8IHx9ASGAkycBsDMPEZGzadA5SiEEJk6caBjAX1JSgilTphjGUVY9f3ldkCSIyEhIv/4qN7/ecAObXomInEyDEmX1aesefPBBk3WqDue4LkRGAvpECZ6jJCJyNg1KlKtWrWqqOByWiIyUb1xLlDxHSUTkXNiZp5GqJ0p9jbKoCHCQq9UQEZEZTJSNZJQodTp4eVU+lp9vm5iIiMh6mCgbq107QKWSq5Dnz8PFBXB1lR9i8ysRkeNjomwstRro0EG+zQ49REROh4nSGqKi5L/s0ENE5HSYKK2hWqLkpANERM6DidIaakmUrFESETk+JkprYKIkInJaTJTWoB8ikpkJ5OUxURIROREmSmvw9QWCguTbaWnszENE5ESYKK1F3/yalsbOPEREToSJ0lqqnKdk0ysRkfNgorQWJkoiIqfERGktVeZ85TlKIiLnwURpLfoa5cmT8PEoB8BESUTkDJgoraVNG3k29LIytCw8DYCdeYiInAETpbUoFIbm14CL8sQDrFESETk+Jkprutb86pPFRElE5CyYKK3pWqL0OCsnyqIioLzclgEREVFjMVFa07VE6ZJ+3FDEWiURkWNjorSma4lSkXYcrq5yERMlEZFjY6K0pk6d5L+5uWjreREAEyURkaNjorQmd3cgPBwA0F3DDj1ERM6AidLarjW/dlExURIROQMmSmu7ligjdXKi5KQDRESOjYnS2q4lyogy1iiJiJwBE6W1XUuUYUVpAJgoiYgcHROltV1LlK0KTsEFpUyUREQOzuaJctmyZYiIiICrqyt69eqF3bt3m12/tLQUs2fPRnh4ODQaDdq3b4+VK1c2U7T1EBgI+PhAIXTogL+YKImIHJzKlk++du1aPP3001i2bBkGDBiA//3vf4iPj8exY8fQpk2bGre59957ceHCBaxYsQIdOnRATk4Oyu1pnjhJkmuVv/6KKBzH1atdbB0RERE1gk0T5ZIlSzB58mQ88sgjAIClS5di27ZtWL58ORYtWmSy/tatW7Fz506cOnUK/v7+AIC2bds2Z8j1ExlpSJSprFESETk0myXKsrIyHDhwADNnzjQqj4uLw969e2vcZvPmzYiJicGbb76Jzz77DB4eHrj99tvx6quvws3NrcZtSktLUVpaarifd60tVKvVQqvVWhy/ftua9qHo2BFKAFE4jn1XddBqKyx+HmsxF689YrxNz9FiZrxN63qMt77b2ixRXrx4ERUVFQgMDDQqDwwMRHZ2do3bnDp1Cnv27IGrqyu++eYbXLx4EQkJCbh06VKt5ykXLVqE+fPnm5Rv374d7u7ujX4dSUlJJmXBhYXoAzlRnj17FYmJuxr9PNZSU7z2jPE2PUeLmfE2resp3qKionqtZ9OmVwCQJMnovhDCpExPp9NBkiR88cUX8PHxASA3395zzz14//33a6xVzpo1CzNmzDDcz8vLQ1hYGOLi4uDt7W1x3FqtFklJSYiNjYVarTZ+MCICeOMNROE4IHwwcuRIi5/HWszGa4cYb9NztJgZb9O6HuPNq2dvS5slyoCAACiVSpPaY05OjkktUy84OBitW7c2JEkA6Ny5M4QQOHfuHDp27GiyjUajgUajMSlXq9VW+TDUuJ+oKAilEl4VBSj8Kwt797bGoEGNfiqrsNbrbi6Mt+k5WsyMt2ldT/HWdzubDQ9xcXFBr169TKrNSUlJ6N+/f43bDBgwAJmZmSgoKDCUnThxAgqFAqGhoU0ab4O4uEBq3x6A3Pz66KNAcbGNYyIiIovYdBzljBkz8PHHH2PlypVITU3FM888g4yMDEyZMgWA3Gw6fvx4w/r3338/WrRogYcffhjHjh3Drl278Pzzz2PSpEm1duaxmWsTD9zkcxwnTwI1nCYlIiIHYNNEOW7cOCxduhQLFixAjx49sGvXLiQmJiL82qWqsrKykJGRYVjf09MTSUlJuHLlCmJiYvDAAw9g9OjRePfdd231Emp3LVE+MkCe83XxYuDgQVsGRERElrB5Z56EhAQkJCTU+Njq1atNyqKiohyjV9a1RNmu7DjGjgXWrQMmTwZ++w1woOZ/IqLrns2nsHNa0dHy33378P5L5+HnB6SkAP/5j02jIiKiBmKibCoxMUCfPkBBAVq+/DjeXiIAAPPmASdO2DY0IiKqPybKpqJUAqtWAS4uwJYtGK/4HHFxQGkp8OijgE5n6wCJiKg+mCibUnQ0MHcuAEB6+il8/GoWPDyAXbuAjz6ycWxERFQvTJRN7fnngZ49gcuXEbYoAa8tFIbic+dsHBsREdWJibKpqdVyE6xaDWzciGmt1qJvXyA/H0hIAISwdYBERGQOE2Vz6NYNmD0bAKCYPhWr/p0DtRr49lvgq69sHBsREZnFRNlcZs2SE2ZuLqLem6rPm5g2DcjNtW1oRERUOybK5uLiIjfBKpXAunV4KWo9unYF/vkH+Ne/5KZYIiKyP0yUzalnT+DaharV0xPw6ZKLcHMDfvgBGDiQnXuIiOwRE2Vze+UVoEsXICcHN65+Cjt3AoGBwB9/ADfdJM/eQ0RE9oOJsrlpNHITrEIB/N//oXfWZuzbJw+5zMyUa5aJibYOkoiI9JgobaF3b3kgJQBMmYK2Ppfx88/ArbcCBQXA6NHA8uW2DZGIiGRMlLYyb558hZGsLGDUKPjmnMD33wMTJ8rT2yUkyLmUU90REdkWE6WtuLoCn3wCeHgAe/cC3bvDZembWPlhORYulFdZvBi4916guNi2oRIRXc+YKG2pTx/gyBEgLg4oKQFefBFS35sw+7YUfPGFPKJk/XpgyBDgxx+BoiJbB0xEdP1horS1tm2BrVuB1asBPz/g4EEgJgb3H52NH74rgZ8f8OuvwLBh8sO33ALMmQPs2MGaJhFRc2CitAeSBEyYABw7BtxzD1BRAbz+OgZO64E/lv2M8eOB1q2BsjJg927g1VeBoUMBX19g0CD5AiW7d3PeWCKipsBEaU+CgoB164ANG+TbaWkIu38gPvGehrN/XMLJk/LluR54AAgJkRPnrl3AggVyTfPmm4Gff7b1iyAici5MlPbozjvl2uXkyXI18b33ILUJQ4elU/HIkL/x+efyLD4nTgAffgjcdx/g5ib3Cbr5ZuCOO4DUVFu/CCIi58BEaa/8/ICPPwaSkoDu3eWePO+/D3TsCNx9N6Rf9qJjR+DRR4H/+z/g5EngkUfkeQw2bQK6dgUee0yexICIiCzHRGnvhg0DDh2SJ4SNj5drmBs2AAMGAP36AV9/DVRUoHVruVn2yBFgzBh5/OVHHwEdOgAvvwxcvWrrF0JE5JiYKB2BJMm9dxIT5Uw4ebI8dmTfPmDsWLmWuXQpcPIkOkcJbNwI7NkD9O8v94x97TUgKkqFzZvbIS/P1i+GiMixMFE6mi5d5CbZjAx5gvUWLYD0dOCZZ4BOnYDQUOD++zHg6IfYsyIN32wQiIoCcnMlrFx5A9q2VSEhQc63RERUNyZKRxUYKHd3zcgAli2TZ1N3cZFPSn75JfD445A6R+GOhBAc7fYv7LrvPQxr9Qu0BaVYvhy44Qa5p+zatXLvWWviMBUiciYqWwdAjeTuDjzxhLwUF8uzEyQnAzt3Ar/8AmRnQ/HVWgzEWiQBEJKEf9zaIKUoEmm7O2H37kh849cJvR+MxLjnwhDapu7fToWFwJkzco4+c8b4dkYGcP484Okp5/LaluBgoHNnwMuryY8QEVGjMFE6Ezc3YPBgeQHkafF++w3YuRO6n35C+W+/waWwEK2KziAOZxCH7fJ6lwH8Fyj+ryvSXDvhpKYrTqi64LiqK44puyJd1xbaCgXKywGtVr7CSVUuKEU4zqAdTuE2nEIbZKDwqgeyrwYh+0QQTiMI+xCECwiEFi5G23boIHfq7dGjcmndukmPEhFRgzBROjNXV7l99ZZbUDFzJr7fsgUj+/SB+tQpeRBmWhp0x08g/+AJuJ3/C26iBJElfyKy5E+j3RTCHccQjaPogqPoAg1KEaU+hUiXU2hbcQotSs5Bgfq1t+ar/fCPMgg5ugBoywRc/iqD+i8tXNaXQQ0ttChDlqSFq0qLm1UStDHdoR7QU86gN94oZ1ZFM5wxqKiQq8dpafKSni5PYN+qlfESGCifJyYip8VEeT2RJKBlS3lan5tvBiCfpPYBgIoKnNpxGld+SYXP2SPwOnMEHqePwu1MKjzKitAbv6M3fq/cl/baoufhAbRvD7RrB7RpI4/7zM6uXC5cALRaeGkvw0t7Ge3MxSmq7H/3dnnR8/SsrILeeKM8V25ZGVBaarqUlMh/JUm+YHZti4uL3F6sT4ppafLA1NLSeh9XVUAAhri5Qblypdym3KkTEBkp/w0IkGOoS0UFcOWKfNvfv37bEFGTY6IkmVKJdrHtgdj2AEZVlpeXA3//DRw9KneVTU2Vm3jbtTNeWrY0/8UuBHD5cmXivHgRUKkAtVpeXFwAtRqlwgWnzqpx9ISEDZ9dgHf6afRACnopDqGH4k+oCwrkefqaYa4+ndoFxaEdcaVVJC75tUdEcAk8i3LkpJ+TIy+5uYAQkP75B96AXAvdvNl4R35+lYkzOFge1Hr5MnDpkvFSdbCrq6vcg1m/hIUZ327RQv5x4u4ur9vQpCqE3I7OnldEdWKiJPNUKvkLPjISuOsuy/cjSXItyd8fiI6udTUNgM4AOmi1UN2YAReXRzBnjgpPHACUunL08U7DS/GHMLzVIaiPHJKTlqtrzTXFa+VCJ6AtKEVpfim0BaUoLyyFrrAEuuJSiGu1z1xlIE4qI3FUG4kDBZE4rI3EGW04dOlKIF2Ozd1dHpHzzDPy7gHIPyRyc6E9fx6/b9qEPn5+UP79t1wrPXFCTpyXL8udrH79tf7Hq6QE+OsveanPsXVzq0yc+kWnq7mmfW1RA7hdkoy3qWlxc5MX/e3qZe7ugLe38eLlJf91cakzfGpmhYXyHJhnzxotyowM9L14EYodO+RWEf3/fXBw3T/EdDr5x29mpvwDUqmUP4/VF3f35jl1YmVMlGS3JAmIjRWIjwe++QZ4+WUVfkntgtFruyAo6EG88grw8MNypa7q/7zhOyBNvp2TI+ezhqrac7ewEPjjD2DWLGDlSuDdd4ERIyD/kAgMBPz9kXP2LHQjR0KpVlfupLhYTnYnTkB3PA3l5y/ApZWf/IPBz6/yx4P+vp+f/KVz/rwcvH7RvzD97StXKpuGhZCbui24YKkkhPziCgsbfoDqQ6ORE6abm/yG1rYoFHJSrenL9dqicHVF1OHDUCQlAfn58jG4erXy79Wrcrm3t9zc3aKF/Lfq7RYt5MeLi+VeafqlsND4vlYrP6+Xl/xBqL7ou2sXFMjPWX0pKIAyLw/9//kHyg8+kH+0ubgYN/frbwshv+cVFTX/1emuvVlS5d+qi55WK5+GqP5Xf/vqVfmzc/lyjW+VAkAgIF/qryovr8oWkchIuSwzE8jKkpfMTPkHa33/ydzc5GPo5ye/Hy1ayJ//6re9veV91vKaFCUl8PD0rN9zNhITJdk9SZIrs2PGAF98IV9W7PRp4Mkn5aW+PD0rvzerLi1aVPbLqbq4u1duK4T83M8/L5++jI+X43n7bSAiovbn1Gnc8POVG/BV8g1Yv17+XomOruycPKi3/Nwm9E3a5pSXy1/4RUXyF70+WepvKxRmz81qJQk/JiZiaP/+UGu1ldtXX/TPUfVv1duFhXJyyMurXPRJu7QU+Oef+r9JZigBRNZnRX1Tto0pALQEgMOHbRxJDby85Cb8Kkt5cDCOHDqEGzQaKE+eBI4flzux5ecDBw7IS11atpSvfKTTVf4A038e9fSfn0Z+LpQAvF94oVH7qC8mSnIYSiUwfjzwr3/JkxO9+qp8ulOlkoeU6E/f6Rf9/aAgOSG6ulr+3JIEPPggcPvt8jwP77wjTz6/bRvw4ovAjBmV61ZUyKdQ162DITlWdeyYvCxbJt/v3LlK4hwkJ+l6UankLzxLB6NqtSj195cTctVasDWUl8u1LX3iLC6Wf21UXXQ649ulpcZfrtUWXUEBzly4gDbdukHp7y9fkNXHx/ivp6f8fBcvyk0NFy+a3s7Pl38F6WuHHh6mNUaVqrKWea2GaLLodJXHX7/oa5teXih3c0PKH3+gR3Q0VBUVlc3e1TufSZL84VYo5L9VbysUlU2VVY9dTfevneeHi0vNtz09K/8pfHxM3jKh1eJMq1boUrVVpLRU7qOg7+R24oR8bIKD5SUkpPJ2YGDtnyOdrvJHlf64Xr4svy+5ufIPm6p/c3Pl416tD0PVvzqVCiUBAdb81NaKiZIcjosLkJAgXzklN1f+EatUNs9ze3sDixcDkyYB06cDP/4IzJ8PfPKJCvHx4di2TYFvvpETuJ6Pj3zps3vuAXr1kqfoTU6Wlz//lPtHpaYCy5fL63fuLM+FP3SonDxr+E6zWHGx/H137Bhw8qQCly+HITQU6NbNyrlSpZITl6+v1XZZodXiz8REhFZv3rZTQqvFeV9fdB850vo/RJqLRiM3gZjpV1AvCkVlM7qVVGi1uJyYaLX9mWPzRLls2TK89dZbyMrKQpcuXbB06VIMHDiwzu1+/vlnDBo0CF27dkVKSkrTB0p2R62Wa4u2EB0tXwFt/Xq5Nnn6tITly3sYHvf1lZPj2LFy0qvap+XOO+UFkBP9rl01J87//lf+funTR06aw4bJF4wxdCSqhRDyD/aTJytrr0ePyn9Pnao85SU3XvXEO+/I8XXtWjnqpkcPOXl6e1vlcBE5NJsmyrVr1+Lpp5/GsmXLMGDAAPzvf/9DfHw8jh07hjZt2tS63dWrVzF+/HgMHToUFy5caMaIiSpJklxLjI8HFi6swJdfFmDwYC+MG6fA0KH16/DZooVp4kxOlmuqP/wgJ7t9++TltdfkfhADB8pDVaueEqy+VCZDU/7+8tz64eE6HDp0CWfPtkBenoSDB037cXTsCIweDYwbB/TuzaGddH2yaaJcsmQJJk+ejEceeQQAsHTpUmzbtg3Lly/HokWLat3u8ccfx/333w+lUomNGzc2U7RENfPwABYs0KFv32SMHDkSarXl3d9btADuvlteAHl0yY8/VibOCxeA7dvN70MvIEBOiPqWM/3tVq3khKfVViAx8WfEx4/EuXNqpKTAsBw6JHewPXkSWLJEXsLDgXvvlZNmz57OmzRLSuQmaj8/W0dC9sJmibKsrAwHDhzAzJkzjcrj4uKwd+/eWrdbtWoV/v77b3z++edYuHBhnc9TWlqK0iozrORduyCjVquFVqutbbM66bdtzD6aE+NtWk0Vb3Cw3InowQflJtVjx4CdOxXIy9MPVxTVhi8KwzDG2k4H6Xvx62MtL9caOkCNHl253sWLwJ49Er7+WoEtWyScOSPhrbeAt94C2rUTuPtuHcaO1aF798YlzaIieYRBQID5U5pN/Zn45x9g2TIFPvhAgdxcCaNG6TBzpg59+lg2KQM/w03LGvHWd1tJCNtMzZGZmYnWrVvj559/Rv/+/Q3lr7/+Oj755BOkpaWZbHPy5EncfPPN2L17Nzp16oR58+Zh48aNZs9Rzps3D/Pnzzcp/7//+z+4V+3/T0S1Ki1V4sCBVtizpzV+/z0QZWWVv7FbtCiGv38JPD3L4OVVBk9PLby8yq4t8m2dDrh0yQ25ua7IzXXDpUuuuHhR/ltQILdRKxQCkZGXEBNzAb16XUB4eF69E7BWKyEjwxvnz3siNDQfbdvm1Xtce2amBzZvbo8dO9qgrMy0V1i3bv/gnntO4IYbLtpVLbqiArh82RU5Oe64dMkVpaUqlJQoUVpauZSUqAy3lUqBkJAChIXlIzS0ACEhBXBxMdNGfx0oKirC/fffj6tXr8LbzAl5m3fmkap98oQQJmUAUFFRgfvvvx/z589Hp06d6r3/WbNmYUaVvvt5eXkICwtDXFyc2QNTF61Wi6SkJMTGxkLtAD3aGG/TcrR4gYbHrD+PWlgosGVLOb7+WoGtWyXk5rohN9etUbG4ugqUlEhITW2B1NQW+OyzaISGCowYoUN8vMCttwq4uMjxDh4cixMn1Dh4UMLBgxIOHJBw+LCEsrLK7w1/f4GBAwUGDRIYNEiHLl1MJ4TZt0/CkiUKbNokQQh52969dZgxQ4foaIElS5T44gsJf/7ZEn/+2RJ9+ujw4os63HabqDEJa7XykMl9+xT49VcJ+/cDBQVF6NTJDWFhEkJDxbVhSwKhoQJt2sg9mvVfd/qhrFWHqZaUSCgokJvgMzLkWr18aTsJZ8/KPxAspVAIREQAkZECUVECHTuWIzPzIHr0uBFKpcoweqf6X29vIDhYICREbp621Y8Ha/zP6VsY62KzRBkQEAClUonsqv3oAeTk5CCwhoFk+fn5+P3333Ho0CFMnToVAKDT6SCEgEqlwvbt23HrrbeabKfRaKCpoZugWq22yheatfbTXBhv03K0eIGGx+zrCzzwgLzk58s9dasOf6s+JE4/9r9168rxrlX/tm4N+PjICeD774EtW4AdO4Bz5yR8/LESH38sd4y6+WYlzp69BRkZbigtNf129vOTJ445cgS4dEnCpk0SNm0CACUCAuShNkOGyOeB//tf4+mCR42SJ5MYOFABSZKz4OrV8pjZxYuBjz4CfvtNgbvvVqBrV3mGpltuAfbvly/7um8f8PvvcoIz5olqX3FGPDzkRFNcLNcQG0qlqhwzrG9ud3ev+W9RkTyHgL5X9ZUrEv7+G/j7bwnyKAslgH4Nen6NRh5KWXUJCpJ7pJeXy0tFheltpVIekzxgQOMTbWP+5+q7nc0SpYuLC3r16oWkpCTcqf+pCiApKQljxowxWd/b2xuHq81wsWzZMuzYsQNff/01IsxNj0JETcLLS/6ys4a2bY2vQZ6cDCQmyokzPR3YsUMBQO5h4+Mjj0mNiZGXXr3kGZLkTkryJDI//STvY88e+Xzr11/Li56Li3zu99lnax8m2KaNPF3h7NnyJBPvvy8n4gceqHl9Pz+gb195iYkpR0rKLwgN7YfMTFX1qVWRm1vzzIH6qXurTqUbGiofn7Zt5U5V+tshIZaNIRZC7himT5qpqcCxYzqkp+fBz88bCoUCCkXl7IL625IkzxiYmSnHX1oqvzfp6Q2P4a235GFIU6cC991nPBOWvbFp0+uMGTPw0EMPISYmBv369cOHH36IjIwMTJkyBYDcbHr+/Hl8+umnUCgU6Nq1q9H2rVq1gqurq0k5ETk2Nzd52E18vJyo0tKAH36owJkzBzFpUg9ERalrrYmo1ZXJatYseSKc/fsrE+fp0/L41unT5c5S9REYCLz+OvDCC3KyXLpUril37SqPbe3bV/7bsWNlE69WK6DVXsLIkaLG+Qb0nZgA43nmNZqmb86UJLnmFxQk17LleCuQmLiz3j23S0rkiTUyM42X7Gy55qhSyYtSWXlbfz87W565KiUFeOQR+bhOniz/SKqrzpOTI/8Q+vVXBcLDGzHdVgPYNFGOGzcOubm5WLBgAbKystC1a1ckJiYiPDwcAJCVlYWMjAxbhkhENiZJQFQU0L69DomJmejQoUeDEomLi1zrHTAAePnlxsXi6yvXLmfOlGtTjakFubvL1yF3VK6ulTVbS7z9tnyBgWXL5BrpW2/JzdyjR8u1zGHD5FrrgQNys7b+79mz+j0o8cILzTOGx+adeRISEpCQkFDjY6tXrza77bx58zBv3jzrB0VEZIZSad9NhY7A3x947jn5snXffy+fN96+Xb6c6+bNcvN61Uu06kmSfDGTnj118PWt58XVG8nmiZKIiK5fSqXcmWrUKLmJ/f335Y5U+iTZqVPl+eheveQpFr299U3FzXOVGCZKIiKyC5GR8jnp116Te+h26mTdiwJYiomSiIjsipeXPLewvbB8UkoiIqLrABMlERGRGUyUREREZjBREhERmcFESUREZAYTJRERkRlMlERERGZcd+Mo9depru91yGqj1WpRVFSEvLw8h7isEuNtWo4WL+B4MTPepnU9xqvPA/q8UJvrLlHm5+cDAMLCwmwcCRER2YP8/Hz4mJkCSBJ1pVIno9PpkJmZCS8vL0iNuJZNXl4ewsLCcPbsWXh7e1sxwqbBeJuWo8ULOF7MjLdpXY/xCiGQn5+PkJAQKBS1n4m87mqUCoUCoaGhVtuft7e3Q3yo9Bhv03K0eAHHi5nxNq3rLV5zNUk9duYhIiIyg4mSiIjIDCZKC2k0GsydOxcajcbWodQL421ajhYv4HgxM96mxXhrd9115iEiImoI1iiJiIjMYKIkIiIyg4mSiIjIDCZKIiIiM5goLbBs2TJERETA1dUVvXr1wu7du20dEgBg3rx5kCTJaAkKCjI8LoTAvHnzEBISAjc3NwwePBhHjx5t1hh37dqF0aNHIyQkBJIkYePGjUaP1yfG0tJSTJs2DQEBAfDw8MDtt9+Oc+fO2STeiRMnmhzzvn372iTeRYsWoXfv3vDy8kKrVq1wxx13IC0tzWgdezq+9YnXno4vACxfvhzdunUzDHLv168fvv/+e8Pj9nR86xOvvR3f6hYtWgRJkvD0008bymxyjAU1yJo1a4RarRYfffSROHbsmHjqqaeEh4eHOHPmjK1DE3PnzhVdunQRWVlZhiUnJ8fw+BtvvCG8vLzE+vXrxeHDh8W4ceNEcHCwyMvLa7YYExMTxezZs8X69esFAPHNN98YPV6fGKdMmSJat24tkpKSxMGDB8WQIUNE9+7dRXl5ebPHO2HCBDFixAijY56bm2u0TnPFO3z4cLFq1Spx5MgRkZKSIm677TbRpk0bUVBQYFjHno5vfeK1p+MrhBCbN28WW7ZsEWlpaSItLU289NJLQq1WiyNHjggh7Ov41ideezu+Vf3222+ibdu2olu3buKpp54ylNviGDNRNlCfPn3ElClTjMqioqLEzJkzbRRRpblz54ru3bvX+JhOpxNBQUHijTfeMJSVlJQIHx8f8cEHHzRThMaqJ576xHjlyhWhVqvFmjVrDOucP39eKBQKsXXr1maNVwj5i2bMmDG1bmPLeHNycgQAsXPnTiGE/R/f6vEKYd/HV8/Pz098/PHHdn98q8crhP0e3/z8fNGxY0eRlJQkBg0aZEiUtjrGbHptgLKyMhw4cABxcXFG5XFxcdi7d6+NojJ28uRJhISEICIiAv/6179w6tQpAEB6ejqys7ONYtdoNBg0aJDdxF6fGA8cOACtVmu0TkhICLp27Wqz15GcnIxWrVqhU6dOePTRR5GTk2N4zJbxXr16FQDg7+8PwP6Pb/V49ez1+FZUVGDNmjUoLCxEv3797P74Vo9Xzx6P75NPPonbbrsNw4YNMyq31TG+7iZFb4yLFy+ioqICgYGBRuWBgYHIzs62UVSVbrrpJnz66afo1KkTLly4gIULF6J///44evSoIb6aYj9z5owtwjVRnxizs7Ph4uICPz8/k3Vs8R7Ex8dj7NixCA8PR3p6Ol555RXceuutOHDgADQajc3iFUJgxowZuPnmm9G1a1cA9n18a4oXsM/je/jwYfTr1w8lJSXw9PTEN998g+joaMOXsL0d39riBezz+K5ZswYHDx7E/v37TR6z1WeYidIC1S/PJYRo1CW7rCU+Pt5w+4YbbkC/fv3Qvn17fPLJJ4YT9PYae1WWxGir1zFu3DjD7a5duyImJgbh4eHYsmUL7rrrrlq3a+p4p06dij///BN79uwxecwej29t8drj8Y2MjERKSgquXLmC9evXY8KECdi5c6fhcXs7vrXFGx0dbXfH9+zZs3jqqaewfft2uLq61rpecx9jNr02QEBAAJRKpcmvkpycHJNfOPbAw8MDN9xwA06ePGno/WrPsdcnxqCgIJSVleHy5cu1rmNLwcHBCA8Px8mTJwHYJt5p06Zh8+bN+Omnn4wuKWevx7e2eGtiD8fXxcUFHTp0QExMDBYtWoTu3bvjnXfesdvjW1u8NbH18T1w4ABycnLQq1cvqFQqqFQq7Ny5E++++y5UKpXhOZv7GDNRNoCLiwt69eqFpKQko/KkpCT079/fRlHVrrS0FKmpqQgODkZERASCgoKMYi8rK8POnTvtJvb6xNirVy+o1WqjdbKysnDkyBG7eB25ubk4e/YsgoODATRvvEIITJ06FRs2bMCOHTsQERFh9Li9Hd+64q2JLY9vbYQQKC0ttbvjW1e8NbH18R06dCgOHz6MlJQUwxITE4MHHngAKSkpaNeunW2OsUVdgK5j+uEhK1asEMeOHRNPP/208PDwEKdPn7Z1aOLZZ58VycnJ4tSpU2Lfvn1i1KhRwsvLyxDbG2+8IXx8fMSGDRvE4cOHxX333dfsw0Py8/PFoUOHxKFDhwQAsWTJEnHo0CHD8Jr6xDhlyhQRGhoqfvjhB3Hw4EFx6623Nll3dXPx5ufni2effVbs3btXpKeni59++kn069dPtG7d2ibxPvHEE8LHx0ckJycbdfcvKioyrGNPx7eueO3t+AohxKxZs8SuXbtEenq6+PPPP8VLL70kFAqF2L59uxDCvo5vXfHa4/GtSdVer0LY5hgzUVrg/fffF+Hh4cLFxUX07NnTqDu7LenHE6nVahESEiLuuusucfToUcPjOp1OzJ07VwQFBQmNRiNuueUWcfjw4WaN8aeffhIATJYJEybUO8bi4mIxdepU4e/vL9zc3MSoUaNERkZGs8dbVFQk4uLiRMuWLYVarRZt2rQREyZMMImlueKtKU4AYtWqVYZ17On41hWvvR1fIYSYNGmS4X+/ZcuWYujQoYYkKYR9Hd+64rXH41uT6onSFseYl9kiIiIyg+coiYiIzGCiJCIiMoOJkoiIyAwmSiIiIjOYKImIiMxgoiQiIjKDiZKIiMgMJkoiIiIzmCiJqN4kScLGjRttHQZRs2KiJHIQEydOhCRJJsuIESNsHRqRU+P1KIkcyIgRI7Bq1SqjMo1GY6NoiK4PrFESORCNRoOgoCCjRX8ld0mSsHz5csTHx8PNzQ0RERFYt26d0faHDx/GrbfeCjc3N7Ro0QKPPfYYCgoKjNZZuXIlunTpAo1Gg+DgYEydOtXo8YsXL+LOO++Eu7s7OnbsiM2bNzftiyayMSZKIifyyiuv4O6778Yff/yBBx98EPfddx9SU1MBAEVFRRgxYgT8/Pywf/9+rFu3Dj/88INRIly+fDmefPJJPPbYYzh8+DA2b96MDh06GD3H/Pnzce+99+LPP//EyJEj8cADD+DSpUvN+jqJmpXF1x0homY1YcIEoVQqhYeHh9GyYMECIYR82aopU6YYbXPTTTeJJ554QgghxIcffij8/PxEQUGB4fEtW7YIhUIhsrOzhRBChISEiNmzZ9caAwDx8ssvG+4XFBQISZLE999/b7XXSWRveI6SyIEMGTIEy5cvNyrz9/c33O7Xr5/RY/369UNKSgoAIDU1Fd27d4eHh4fh8QEDBkCn0yEtLQ2SJCEzMxNDhw41G0O3bt0Mtz08PODl5YWcnBxLXxKR3WOiJHIgHh4eJk2hdZEkCQAghDDcrmkdNze3eu1PrVabbKvT6RoUE5Ej4TlKIieyb98+k/tRUVEAgOjoaKSkpKCwsNDw+M8//wyFQoFOnTrBy8sLbdu2xY8//tisMRPZO9YoiRxIaWkpsrOzjcpUKhUCAgIAAOvWrUNMTAxuvvlmfPHFF/jtt9+wYsUKAMADDzyAuXPnYsKECZg3bx7++ecfTJs2DQ899BACAwMBAPPmzcOUKVPQqlUrxMfHIz8/Hz///DOmTZvWvC+UyI4wURI5kK1btyI4ONioLDIyEsePHwcg90hds2YNEhISEBQUhC+++ALR0dEAAHd3d2zbtg1PPfUUevfuDXd3d9x9991YsmSJYV8TJkxASUkJ3n77bTz33HMICAjAPffc03wvkMgOSUIIYesgiKjxJEnCN998gzvuuMPWoRA5FZ6jJCIiMoOJkoiIyAyeoyRyEjyLQtQ0WKMkIiIyg4mSiIjIDCZKIiIiM5goiYiIzGCiJCIiMoOJkoiIyAwmSiIiIjOYKImIiMz4fwtNjSszXELfAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 500x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# 创建图形\n",
    "plt.figure(figsize=(5, 3))\n",
    "\n",
    "plt.plot(x_axis_epoch, y_axis_train_error, label='Train Error', color='blue')\n",
    "plt.plot(x_axis_epoch, y_axis_test_error, label='Test Error', color='red')\n",
    "\n",
    "# 添加标题和标签\n",
    "plt.title('Train Error and Test Error')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Error')\n",
    "\n",
    "# 添加图例\n",
    "plt.grid(True)\n",
    "plt.legend()\n",
    "\n",
    "# 显示图形\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da35bcd1",
   "metadata": {},
   "source": [
    "# 激活函数\n",
    "用tanh函数作为中间层激活函数\n",
    "用Softmax函数作为输出层的激活函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "7965addc",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys, numpy as np\n",
    "from keras.datasets import mnist\n",
    "\n",
    "# load the MNIST dataset\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "\n",
    "# 训练数据一维化，归一化\n",
    "images, labels = (x_train[0:1000].reshape(1000,28*28) / 255, y_train[0:1000])\n",
    "# 训练数据one-hot编码\n",
    "one_hot_labels = np.zeros((len(labels),10))\n",
    "for i,label in enumerate(labels):\n",
    "    one_hot_labels[i][label] = 1\n",
    "labels = one_hot_labels\n",
    "\n",
    "# 测试数据归一化\n",
    "test_images = x_test.reshape(len(x_test),28*28) / 255  \n",
    "# 测试数据one-hot编码\n",
    "test_labels = np.zeros((len(y_test),10))\n",
    "for i,label in enumerate(y_test):\n",
    "    test_labels[i][label] = 1\n",
    "\n",
    "np.random.seed(1)\n",
    "\n",
    "def tanh(x):\n",
    "    return np.tanh(x)\n",
    "\n",
    "def tanh2deriv(output):\n",
    "    return 1 - (output ** 2)\n",
    "\n",
    "def softmax(x):\n",
    "    temp = np.exp(x)\n",
    "    return temp / np.sum(temp, axis=1, keepdims=True)\n",
    "\n",
    "batch_size = 100\n",
    "alpha, epochs, hidden_size, pixels_per_image, num_labels = (2, 350, 100, 784, 10)\n",
    "\n",
    "# 权重初始化\n",
    "weights_0_1 = 0.02*np.random.random((pixels_per_image,hidden_size)) - 0.01\n",
    "weights_1_2 = 0.2*np.random.random((hidden_size,num_labels)) - 0.1\n",
    "\n",
    "# 可视化数据记录\n",
    "x_axis_epoch = []\n",
    "y_axis_train_error = []\n",
    "y_axis_test_error = []\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "c45db51f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I:0 Train-Err:0.897 Train-Acc:0.156 Test-Err:0.896 Test-Acc:0.302\n",
      "I:10 Train-Err:0.803 Train-Acc:0.723 Test-Err:0.856 Test-Acc:0.6833\n",
      "I:20 Train-Err:0.660 Train-Acc:0.732 Test-Err:0.790 Test-Acc:0.6948\n",
      "I:30 Train-Err:0.525 Train-Acc:0.763 Test-Err:0.716 Test-Acc:0.7278\n",
      "I:40 Train-Err:0.424 Train-Acc:0.794 Test-Err:0.652 Test-Acc:0.7576\n",
      "I:50 Train-Err:0.367 Train-Acc:0.819 Test-Err:0.600 Test-Acc:0.7843\n",
      "I:60 Train-Err:0.320 Train-Acc:0.849 Test-Err:0.556 Test-Acc:0.7981\n",
      "I:70 Train-Err:0.282 Train-Acc:0.864 Test-Err:0.521 Test-Acc:0.8111\n",
      "I:80 Train-Err:0.255 Train-Acc:0.867 Test-Err:0.493 Test-Acc:0.8155\n",
      "I:90 Train-Err:0.230 Train-Acc:0.885 Test-Err:0.469 Test-Acc:0.8208\n",
      "I:100 Train-Err:0.215 Train-Acc:0.883 Test-Err:0.449 Test-Acc:0.8278\n",
      "I:110 Train-Err:0.203 Train-Acc:0.891 Test-Err:0.429 Test-Acc:0.8321\n",
      "I:120 Train-Err:0.196 Train-Acc:0.901 Test-Err:0.411 Test-Acc:0.8346\n",
      "I:130 Train-Err:0.179 Train-Acc:0.901 Test-Err:0.399 Test-Acc:0.8372\n",
      "I:140 Train-Err:0.172 Train-Acc:0.905 Test-Err:0.387 Test-Acc:0.8406\n",
      "I:150 Train-Err:0.160 Train-Acc:0.914 Test-Err:0.377 Test-Acc:0.8413\n",
      "I:160 Train-Err:0.149 Train-Acc:0.925 Test-Err:0.368 Test-Acc:0.8431\n",
      "I:170 Train-Err:0.147 Train-Acc:0.918 Test-Err:0.359 Test-Acc:0.8443\n",
      "I:180 Train-Err:0.136 Train-Acc:0.933 Test-Err:0.349 Test-Acc:0.8471\n",
      "I:190 Train-Err:0.138 Train-Acc:0.933 Test-Err:0.341 Test-Acc:0.8469\n",
      "I:200 Train-Err:0.133 Train-Acc:0.926 Test-Err:0.335 Test-Acc:0.8478\n",
      "I:210 Train-Err:0.123 Train-Acc:0.931 Test-Err:0.330 Test-Acc:0.848\n",
      "I:220 Train-Err:0.124 Train-Acc:0.93 Test-Err:0.324 Test-Acc:0.8479\n",
      "I:230 Train-Err:0.116 Train-Acc:0.937 Test-Err:0.317 Test-Acc:0.8491\n",
      "I:240 Train-Err:0.114 Train-Acc:0.938 Test-Err:0.310 Test-Acc:0.8504\n",
      "I:250 Train-Err:0.111 Train-Acc:0.937 Test-Err:0.308 Test-Acc:0.8484\n",
      "I:260 Train-Err:0.108 Train-Acc:0.945 Test-Err:0.302 Test-Acc:0.8473\n",
      "I:270 Train-Err:0.099 Train-Acc:0.951 Test-Err:0.297 Test-Acc:0.8479\n",
      "I:280 Train-Err:0.102 Train-Acc:0.949 Test-Err:0.295 Test-Acc:0.8464\n",
      "I:290 Train-Err:0.100 Train-Acc:0.94 Test-Err:0.292 Test-Acc:0.8451\n",
      "I:300 Train-Err:0.091 Train-Acc:0.95 Test-Err:0.288 Test-Acc:0.8447\n",
      "I:310 Train-Err:0.094 Train-Acc:0.952 Test-Err:0.287 Test-Acc:0.8437\n",
      "I:320 Train-Err:0.091 Train-Acc:0.95 Test-Err:0.286 Test-Acc:0.8426\n",
      "I:330 Train-Err:0.093 Train-Acc:0.944 Test-Err:0.287 Test-Acc:0.8392\n",
      "I:340 Train-Err:0.090 Train-Acc:0.952 Test-Err:0.286 Test-Acc:0.8361\n",
      "I:349 Train-Err:0.084 Train-Acc:0.956 Test-Err:0.291 Test-Acc:0.8285\n"
     ]
    }
   ],
   "source": [
    "\n",
    "for j in range(epochs):\n",
    "    error, correct_cnt = (0.0, 0)  # 训练集误差，预测正确个数计数\n",
    "    \n",
    "    weight_update_num = int(len(images) / batch_size) # 每轮迭代更新权重次数\n",
    "    for i in range(weight_update_num):  # BGD\n",
    "        batch_start, batch_end = ((i * batch_size),((i+1)*batch_size))\n",
    "\n",
    "        # 前向传播\n",
    "        layer_0 = images[batch_start:batch_end]\n",
    "        layer_1 = tanh(np.dot(layer_0,weights_0_1))\n",
    "        dropout_mask = np.random.randint(2, size=layer_1.shape) # 模拟dropout\n",
    "        layer_1 *= dropout_mask * 2                             # *2是为了保证在训练时保持神经元输出的期望不变\n",
    "        layer_2 = softmax(np.dot(layer_1,weights_1_2))\n",
    "\n",
    "        error += np.sum((labels[batch_start:batch_end] - layer_2) ** 2)  # 均方差\n",
    "        for k in range(batch_size):\n",
    "            correct_cnt += int(np.argmax(layer_2[k:k+1]) == \\\n",
    "                               np.argmax(labels[batch_start+k:batch_start+k+1])) # 预测正确个数计数\n",
    "\n",
    "        # 反向传播\n",
    "        layer_2_delta = (labels[batch_start:batch_end]-layer_2) / (batch_size * layer_2.shape[0])\n",
    "        layer_1_delta = layer_2_delta.dot(weights_1_2.T) * tanh2deriv(layer_1)\n",
    "        layer_1_delta *= dropout_mask\n",
    "        \n",
    "        # 更新权重\n",
    "        weights_1_2 += alpha * layer_1.T.dot(layer_2_delta)\n",
    "        weights_0_1 += alpha * layer_0.T.dot(layer_1_delta)\n",
    "\n",
    "    # 每隔10次迭代输出结果，并记录测试集准确率\n",
    "    if(j % 10 == 0 or j == epochs-1):\n",
    "        \n",
    "        # 测试集\n",
    "        test_error, test_correct_cnt = (0.0, 0)   # 测试集误差，预测正确个数计数\n",
    "\n",
    "        for i in range(len(test_images)):\n",
    "\n",
    "            layer_0 = test_images[i:i+1]\n",
    "            layer_1 = relu(np.dot(layer_0,weights_0_1))\n",
    "            layer_2 = softmax(np.dot(layer_1,weights_1_2))\n",
    "            \n",
    "            test_error += np.sum((test_labels[i:i+1] - layer_2) ** 2)\n",
    "            test_correct_cnt += int(np.argmax(layer_2) == \\\n",
    "                                            np.argmax(test_labels[i:i+1]))\n",
    "        \n",
    "        sys.stdout.write(\"I:\"+str(j)+ \\\n",
    "                     \" Train-Err:\" + str(error/float(len(images)))[0:5] +\\\n",
    "                     \" Train-Acc:\" + str(correct_cnt/float(len(images))))\n",
    "        sys.stdout.write(\" Test-Err:\" + str(test_error/float(len(test_images)))[0:5] +\\\n",
    "                         \" Test-Acc:\" + str(test_correct_cnt/float(len(test_images))))\n",
    "        print()  # 换行\n",
    "\n",
    "        x_axis_epoch.append(j) # 记录轮次\n",
    "        y_axis_train_error.append(error/float(len(images))) # 记录训练集误差\n",
    "        y_axis_test_error.append(test_error/float(len(test_images))) # 记录测试集误差\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "2cf3fa52",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcoAAAE6CAYAAACWDhLFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABW8klEQVR4nO3dd3gUVdvA4d9uek8gkAIhhN4EJfQiTSIBFEWUDwWpKkRABPEFUSkiqK8iFsqrAgqiIIiogECkF1FEerMAhhYiNZCQuvP9cdxNliSbtsluNs99XXPt7OzM7HMygSdz5hSdpmkaQgghhMiV3tYBCCGEEPZMEqUQQghhgSRKIYQQwgJJlEIIIYQFkiiFEEIICyRRCiGEEBZIohRCCCEskEQphBBCWCCJUgghhLBAEqUoUTqdrkDL1q1bi/U9U6ZMQafTWSfof1mKd9CgQVb9rrIov5/51q1bC3z9reHYsWNMmTKFM2fOFGj/Tz/9tER/J4XjcLZ1AMKx/fTTT2bvX3vtNbZs2cLmzZvNtjdo0KBY3zNs2DC6detWrHPkpk+fPowbNy7H9kqVKln9uxxN06ZNc1z/hx9+mJo1a/L2229b/fuOHTvG1KlT6dixI9WrVy/wcYsWLaJevXo5thf3d1I4DkmUokS1atXK7H2lSpXQ6/U5tt8pOTkZT0/PAn9P1apVqVq1apFitCQoKCjfWHOTV/yappGSkoKHh0eRY7p9+zbu7u5Wv4O2Nl9f3xw/Ozc3N/z9/Yv0My0pjRo1olmzZoU6xtJ1tMb1KezvvyhZUvUqbK5jx440atSI7du306ZNGzw9PRkyZAgAy5cvJyoqipCQEDw8PKhfvz4TJkwgKSnJ7By5VQNWr16dnj17sn79epo2bYqHhwf16tVj4cKFVo1/0KBBeHt7c/jwYaKiovDx8aFLly6Aqr4dOXIk8+fPp379+ri5ufHZZ58BsHPnTrp06YKPjw+enp60adOGtWvXmp3bWD24ceNGhgwZQqVKlfD09CQ1NTXXWFJSUhg3bhx33303fn5+VKhQgdatW/Ptt9/m2NcY25IlS6hfvz6enp40adKENWvW5Nh37dq13H333bi5uREREWHVO8L4+HieeeYZqlatiqurKxEREUydOpWMjAyz/ebNm0eTJk3w9vbGx8eHevXq8dJLLwHq5/Too48C0KlTJ1P16aeffmqVGPO6jpauj8Fg4K233qJevXq4ublRuXJlnnzySc6dO2d2bku//8I+yB2lsAsXL16kf//+vPjii8yYMQO9Xv0N98cff9C9e3fGjBmDl5cXJ06c4M033+SXX37JUX2bm4MHDzJu3DgmTJhAUFAQn3zyCUOHDqVWrVrce++9+R6vaVqO/7ABnJyczBJzWloaDz74IM888wwTJkwwO2b16tXs2LGDV199leDgYCpXrsy2bdvo2rUrjRs3ZsGCBbi5uTF37lweeOABvvzyS/r27Wv2fUOGDKFHjx4sWbKEpKQkXFxcco03NTWVq1ev8sILL1ClShXS0tL48ccf6d27N4sWLeLJJ58023/t2rXs3buXadOm4e3tzVtvvcXDDz/MyZMnqVGjBgCbNm2iV69etG7dmmXLlpGZmclbb73FpUuX8v355Sc+Pp4WLVqg1+t59dVXqVmzJj/99BPTp0/nzJkzLFq0CIBly5YRExPDqFGjePvtt9Hr9fz5558cO3YMgB49ejBjxgxeeukl5syZQ9OmTQGoWbNmvjFkZmbmuMY6nQ4nJyezbbldx7179wK5X58RI0bw0UcfMXLkSHr27MmZM2d45ZVX2Lp1K7/99huBgYGmc+f1+y/shCZEKRo4cKDm5eVltq1Dhw4aoG3atMnisQaDQUtPT9e2bdumAdrBgwdNn02ePFm789c5PDxcc3d31/7++2/Tttu3b2sVKlTQnnnmmXxjBfJclixZYlYmQFu4cGGu5/Dz89OuXr1qtr1Vq1Za5cqVtZs3b5q2ZWRkaI0aNdKqVq2qGQwGTdM0bdGiRRqgPfnkk/nGm5uMjAwtPT1dGzp0qHbPPffkiC0oKEhLTEw0bYuPj9f0er02c+ZM07aWLVtqoaGh2u3bt03bEhMTtQoVKuT4mecnPDxc69Gjh+n9M888o3l7e5tdI03TtLffflsDtKNHj2qapmkjR47U/P39LZ57xYoVGqBt2bKlQLEYf7a5LU5OTmb75nUd87o+x48f1wAtJibGbPvPP/+sAdpLL71k2lbQ339hO/Jni7ALAQEBdO7cOcf2U6dO8fjjjxMcHIyTkxMuLi506NABgOPHj+d73rvvvptq1aqZ3ru7u1OnTh3+/vvvAsX12GOPsXfv3hxL9+7dc+z7yCOP5HqOzp07ExAQYHqflJTEzz//TJ8+ffD29jZtd3JyYsCAAZw7d46TJ08W6Ny5WbFiBW3btsXb2xtnZ2dcXFxYsGBBrj+vTp064ePjY3ofFBRE5cqVTT+fpKQk9u7dS+/evXF3dzft5+PjwwMPPFDgmPKyZs0aOnXqRGhoKBkZGaYlOjoagG3btgHQokULrl+/Tr9+/fj222+5fPlysb/baPHixTmu788//5xjvzuvY3Z3Xp8tW7YA5Ggd3aJFC+rXr8+mTZvMtuf1+y/sg1S9CrsQEhKSY9utW7do37497u7uTJ8+nTp16uDp6cnZs2fp3bs3t2/fzve8FStWzLHNzc2tQMeCanxUkIYenp6e+Pr65vrZnWW7du0amqblWubQ0FAArly5YvEceVm1ahWPPfYYjz76KOPHjyc4OBhnZ2fmzZuX67PZ/H4+165dw2AwEBwcnGO/3LYV1qVLl/j+++/zrEo2JsQBAwaQkZHBxx9/zCOPPILBYKB58+ZMnz6drl27FiuG+vXrF+gaW7oGd35mvH55XeM7/1Ar6PUVtiGJUtiF3FoIbt68mQsXLrB161bTXSTA9evXSzGygrHUwvHOzwICAtDr9Vy8eDHHvhcuXAAwe36V3/mz+/zzz4mIiGD58uVmx+TV+Cc/AQEB6HQ64uPjc3yW27bCCgwMpHHjxrz++uu5fm78wwFg8ODBDB48mKSkJLZv387kyZPp2bMnv//+O+Hh4cWOJT+FucbGP0AuXryYozX2hQsXinx9hW1I1auwW8b/PNzc3My2/+9//7NFOFbj5eVFy5YtWbVqldmdrcFg4PPPP6dq1arUqVOnSOfW6XS4urqa/ccbHx+fa6vXgsbaokULVq1aRUpKimn7zZs3+f7774t0zux69uzJkSNHqFmzJs2aNcuxZE+U2WOKjo5m0qRJpKWlcfToUSDr96SgtQUlyViN+vnnn5tt37t3L8ePHze1ihZlg9xRCrvVpk0bAgICGD58OJMnT8bFxYWlS5dy8ODBUovh0qVL7NmzJ8d2X1/fYnVInzlzJl27dqVTp0688MILuLq6MnfuXI4cOcKXX35Z5DuMnj17smrVKmJiYujTpw9nz57ltddeIyQkhD/++KNI53zttdfo1q0bXbt2Zdy4cWRmZvLmm2/i5eXF1atXi3ROo2nTphEbG0ubNm0YPXo0devWJSUlhTNnzrBu3Trmz59P1apVeeqpp/Dw8KBt27aEhIQQHx/PzJkz8fPzo3nz5oDqDwnw0Ucf4ePjg7u7OxEREblWL2d35MiRXFs216xZs8gDS9StW5enn36aDz74AL1eT3R0tKnVa1hYGM8//3yRzitsQxKlsFsVK1Zk7dq1jBs3jv79++Pl5UWvXr1Yvny5qfl/SVu5ciUrV67Msb1t27bs3LmzyOft0KEDmzdvZvLkyQwaNAiDwUCTJk347rvv6NmzZ5HPO3jwYBISEpg/fz4LFy6kRo0aTJgwgXPnzjF16tQinbNr166sXr2al19+mb59+xIcHExMTAy3b98u8jmNQkJC+PXXX3nttdf473//y7lz5/Dx8SEiIoJu3bqZGs+0b9+eTz/9lK+++opr164RGBhIu3btWLx4sSmZRUREMHv2bN577z06duxIZmYmixYtyne4wcGDB+e6/eOPP2bYsGFFLtu8efOoWbMmCxYsYM6cOfj5+dGtWzdmzpyZb/IW9kWnaZpm6yCEEEIIeyXPKIUQQggLJFEKIYQQFkiiFEIIISyQRCmEEEJYIIlSCCGEsEASpRBCCGFBuetHaTAYuHDhAj4+PjJslBBClGOapnHz5k1CQ0MtTm1W7hLlhQsXCAsLs3UYQggh7MTZs2dzjMmbXblLlMYphc6ePZvnbA8FkZ6ezsaNG4mKispz5oOywFHKAY5TFkcpBzhOWRylHOA4ZbFGORITEwkLCzObai435S5RGqtbfX19i50ojVMrlfVfNkcoBzhOWRylHOA4ZXGUcoDjlMWa5cjvMZw05hFCCCEskEQphBBCWCCJUgghhLCg3D2jFEKI7DRNIyMjg8zMzDz3SU9Px9nZmZSUFIv7lQWOUpaClMPJyQlnZ+didwWURCmEKLfS0tK4ePEiycnJFvfTNI3g4GDOnj1b5vtfO0pZCloOT09PQkJCcHV1LfJ3SaIsKk0jLcXWQQghispgMHD69GmcnJwIDQ3F1dU1z/9wDQYDt27dwtvb22LH9LLAUcqSXzk0TSMtLY1//vmH06dPU7t27SKXVxJlEfx92sDhTs9juJyI1qsblN0W1kKUW2lpaRgMBsLCwvD09LS4r8FgIC0tDXd39zKdXMBxylKQcnh4eODi4sLff/9t2rcoJFEWQcUz+4j+ez5OGIjr40m17+dDGa7CEKI8K8vJQuTPGtdXfkOKwLtTcz7r/CkGdFRb+xGMGQOaZuuwhBBClABJlEXU9J3/YwgL1Zv334cXX5RkKYQQDkgSZRE1bAj7GvXgaf6nNrz9Nrz8siRLIUSZ07FjR8aMGWPrMOyWJMpi6N79NB/zNBO9P1AbZsyA116zbVBCCIel0+ksLoMGDSrSeVetWsVrxfy/a9CgQbnG1K1bt2Kd1x5IY55iaNEinipVNN44P5I+T6QTuXQsTJ4Mrq4wYYKtwxNCOJiLFy+a1pcvX86rr77KyZMnTds8PDzM9k9PTy/QgOEVKlSwSnzdunVj0aJFZtvc3Nzy3D+3+Aoac0HOZS1yR1kMzs4aw4YZABh16nmYOVN9MHEizJplw8iEEEWhaZCUVPpLQZ/YBAcHmxY/Pz90Op3pfUpKCv7+/nz11Vd07NgRd3d3Pv/8c65cuUK/fv2oWrUqnp6eNGnShJUrV5qd986q1+rVqzNjxgyGDBmCj48P1apV46OPPso3Pjc3N7MYg4ODCQgIMH2u0+mYP38+vXr1wsvLi+nTpzNlyhTuvvtuFi5cSI0aNXBzc0PTNOLi4ujVqxfe3t74+vry2GOPcenSJdO5pk6dSvv27XMcVxIkURbT0KEGXFzgp59g//0TYOpU9cG4cfDhh7YNTghRKMnJ4O2dc/H11VO1qj++vvpcPy/uks/AQIXyn//8h9GjR3P8+HHuv/9+UlJSiIyMZM2aNRw5coSnnnqK4cOH8/PPP1s8zzvvvEOzZs3Yv38/MTExjBgxghMnThQ7vsmTJ9OrVy8OHz7MkCFDAPjzzz/56quv+Prrrzlw4AAADz30EFevXmXbtm3Exsby119/0bdvX7NznT59mhUrVpgdVxKk6rWYgoPhkUdg2TKYMwc++fgVSEuD11+HUaNUNezTT9s6TCFEOTFmzBh69+5ttu2FF14wrY8cOZI1a9awcuVKWrduned5unfvTkxMDKCS77vvvsvWrVupV69ensesWbMGb29vs23/+c9/eOWVV0zvH3/8cVOCNEpLS2PJkiVUqlQJgNjYWA4dOsTp06cJCwsDYMmSJTRs2JC9e/fSvHlz03GLFy8mKCgoz5isQRKlFTz7rEqUX3wBb72lo8Jrr0FqqmoJ+8wz4OICgwfbOkwhRD48PeHWrZzbDQYDiYmJ+Pr6lsgABfkMDFQozZo1M3ufmZnJG2+8wfLlyzl//jypqamkpqbi5+dn8TyNGzc2rRureBMSEiwe06lTJ+bNm2e27c7nn3fGBxAeHm5KkgDHjx8nLCzMlCQBGjRogL+/P8ePHzclyrCwMLPjSookSito2xYaN4ZDh2DRIhg3TgdvvaXuLN9/H4YNUztERto6VCGEBTodeHnl3G4wQGam+szeB/LxuqMA77zzDu+++y6zZ8/mrrvuwsPDg1GjRpGWlmbxPHc2jNHpdBgMhny/u1atWoWKL7dtmqblOu7undvzG3rQWuz8kpcNOp26qwSYN0/9o0Kng9mzoU8ftWHUqH8/EEKI0rNjxw569epF//79adKkCTVq1ODUqVO2DsuiBg0aEBcXx9mzZ03bjh07xo0bN6hfv36pxyOJ0kqeeAL8/OCvv2DDhn83GpOll5dq7fP557YMUQhRDtWqVYvY2Fh2797N8ePHGT58uFnrUWtKTU0lPj7ebLl8+XKhz3PffffRuHFjnnjiCX777Td++eUXnnzySTp06JBr1W1Jk0RpJV5eWY8h58zJ9kGVKmB8kP3ii5CYWOqxCSHKr1deeYWmTZty//3307FjR4KDg+nRo0eJfNf69esJCQkxW9q1a1fo8+h0OlavXk1AQAD33nsv9913HzVq1GD58uUlEHUBaOXMjRs3NEC7ceNGsc6TlpamrV69WktLSzNt+/13TQNN0+k07dSpbDunpGha7drqw3HjivW91pZbOcoqRymLo5RD0+y7LLdv39aOHTum3b59O999MzMztWvXrmmZmZmlEFnJcpSyFLQclq5zQfOB3FFaUe3aEBWlOg+bNfxyc4P33lPr770Hx4/bJD4hhBCFJ4nSyoyNehYsgNu3s30QHQ09e0JGBjz3nAyeLoQQZYQkSivr0QPCw+HqVdW30sy776oBCGJjYfVqW4QnhBCikCRRWpmTE4wYodbnzLnjxrFWLTCOkDF27B23nEIIIeyRJMoSMHSoeiy5bx/88ssdH770ElStCmfOqEEJhBBC2DVJlCUgMBCMY/eadRUB1Y/k7bfV+htvqIQphBDCbtk8Uc6dO5eIiAjc3d2JjIxkx44dFvdfunQpTZo0wdPTk5CQEAYPHsyVK1dKKdqCMzbqWb4ccgyP+Nhj0LEjpKSoWUaEEELYLZsmyuXLlzNmzBgmTZrE/v37ad++PdHR0cTFxeW6/86dO3nyyScZOnQoR48eZcWKFezdu5dhw4aVcuT5a9ECmjVTw70uWHDHhzqdGgPWyQlWrYIff7RJjEIIIfJn00Q5a9Yshg4dyrBhw6hfvz6zZ88mLCwsx+jzRnv27KF69eqMHj2aiIgI2rVrxzPPPMOvv/5aypEXjPGucv58NaCymbvugn+nsGH0aEhPL9XYhBBCFIzNZg9JS0tj3759TJgwwWx7VFQUu3fvzvWYNm3aMGnSJNatW0d0dDQJCQmsXLnS4nBMxilljBL/HUIuPT2d9GIkJ+Oxls7RuzeMHetMXJyO7dszaNfujr6TL7+M85dfojt+nMz33sPw3HNFjqeoClKOssJRyuIo5QD7Lkt6ejqapmEwGPKdFUP7t/m6cf+yzFHKUtByGAwGNE0jPT0dJycns88K+ntps0R5+fJlMjMzc0y4GRQURHx8fK7HtGnThqVLl9K3b19SUlLIyMjgwQcf5IMPPsjze2bOnMnUqVNzbN+4caNVpmiJjY21+HnDhpHs3FmV+fP/IjEx5+zg1fr25Z45czBMnsymSpVI9fcvdkxFkV85yhJHKYujlAPssyzOzs4EBwdz69atfKecMrp582YJR2VZQECAxc/79evH3LlzC3SuO8vSuHFjRowYwQhj/7Y8NG7c2GxWD6NXX32V559/vkDfbU35XZO0tDRu377N9u3bycjIMPssOTm5QN9h8/ko75xzTMtjHjJQ06yMHj2aV199lfvvv5+LFy8yfvx4hg8fzoIcDwKViRMnMnbsWNP7xMREwsLCiIqKwtfXt8hxp6enExsbS9euXXPM25ZdQoKOnTvhzJnadO9eI+cO3bph2LMHl337iNq8mcxPPilyTEVR0HKUBY5SFkcpB9h3WVJSUjh79ize3t64u7tb3FfTNG7evImPj0+e/z+VhvPnz5vWv/rqKyZPnszxbENienh45Pv/Wl5l0ev1uLu753u8Xq9n6tSpOdqG+Pj45DrXpKZpZGZm4uxsnm7S0tJwdXW1+F25MR5X0GuSkpKCh4cH9957b47rnFjASSpsligDAwNxcnLKcfeYkJCQ4y7TaObMmbRt25bx48cD6i8bLy8v2rdvz/Tp0wkJCclxjJubG25ubjm2u7i4WOUfbn7n6dZNvf76q55bt/Tk+gfhnDnQqhX6xYvRjxwJ/87eXZqs9fOwB45SFkcpB9hnWTIzM9HpdOj1evTG2Zg1DXK5yzAYDJCUhM7JKWtfa/L0VI388hEaGmpa9/f3R6fTmW37/vvvmTJlCkePHiU0NJSBAwcyadIkU5KaMmUKCxcu5NKlS1SsWJE+ffrw/vvv07FjR/7++2/Gjh1rurHQLAyz6evra/a92W3dupVOnTqxfv16Jk2axKFDh9iwYQNTp06lUaNGuLq6snjxYho2bMi2bdvYtm0b48eP5+DBg1SoUIGBAwcyffp0U8wdO3bM9ThjdavxGuZFr9ej0+ly/R0s6O+kzRKlq6srkZGRxMbG8vDDD5u2x8bG0qtXr1yPSU5OzvFXibHO2dJFtaWwMKhXD06cgM2b4ZFHctmpZUsYMACWLFEDEthhNZUQ5UJyMnh759isB/xL8ntv3VJ9rIthw4YN9O/fn/fff5/27dvz119/8fTTTwMwefJkVq5cybvvvssXX3xBtWrVSEpK4vDhwwCsWrWKJk2a8PTTT/PUU08VuzgAL774Im+//TY1atTA/99HSp999hkjRoxg165daJrG+fPn6d69O4MGDWLx4sWcOHGCp556Cnd3d6ZMmWI6153HlbrCTGtibcuWLdNcXFy0BQsWaMeOHdPGjBmjeXl5aWfOnNE0TdMmTJigDRgwwLT/okWLNGdnZ23u3LnaX3/9pe3cuVNr1qyZ1qJFiwJ/Z0lOs5WX0aPVDFtPP21hp9OnNc3FRe24aVOxYisMe54GqbAcpSyOUg5Ns++y5Dr90q1b6t9gaS+3bhU6/kWLFml+fn6m9+3bt9dmzJhhts+SJUu0kJAQTdM07Z133tHq1KmjpaSk5Do9VXh4uPbuu+/m+73h4eGaq6ur5uXlZbZs2bJF0zRN27JliwZoq1evNjuuQ4cO2t1332227aWXXtLq1q2rGQwG07Y5c+Zo3t7epvhyO07TSneaLZs+o+zbty9Xrlxh2rRpXLx4kUaNGrFu3TrCw8MBuHjxolmfykGDBnHz5k0+/PBDxo0bh7+/P507d+bNN9+0VREKJCpKdZvcuFH9q8i1hqV6dXjmGfjwQ5g4EfbsKVBVjBDCijw91d3dHQwGA4mJifj6+pZc1Wsx7du3j7179/L666+btmVmZpKSkkJycjKPPvoos2fPplatWnTu3JkHH3yQXr165ailK4jx48czaNAgs21VqlQxe9+sWbMcx9257fjx47Ru3drsGWPbtm25desW586do1q1anmeqzTZvDFPTEwMMcb+hHf49NNPc2wbNWoUo0aNKuGorKtDB3BxUaPV/fWXGhs9V5MmwcKFaoDYb7+Fhx4qxSiFEOh0uVeBGgyqM7SXF5REorQCg8HA1KlT6d27d47P3N3dCQsL4+TJk2zYsIEffviBkSNH8s4777Bt27ZCPz8ODAykVp7/kSm5Ney5c5uWS+NN7d+q1ezbcztXabLPK+5gvL2hTRu1vnGjhR2Dg9VclaCSZo5RCoQQIndNmzbl5MmT1KpVK8divAv28PDgwQcf5M0332Tz5s389NNPpueUrq6uZJby/zkNGjRg9+7dZs8dd+/ejY+PT447VFuSRFlKoqLUq8VECTB+PPj7w7FjsHRpSYclhHAQr776KosXLza1ej1+/DjLly/n5ZdfBlQN3YIFCzhy5Ahnzpzh888/x8PDw/Soq3r16mzfvp3z589z+fJli9918+ZN4uPjzZaCdrXILiYmhrNnzzJq1ChOnDjBt99+y+TJkxk7dmzJVHEXkf1E4uCMiXLLlnxGqwsIgP/8R61PnqwGixVCiHzcf//9rFmzhtjYWJo3b06rVq2YNWuWKRH6+/vz8ccf0759e9q1a8fmzZv5/vvvqVixIgDTpk3jzJkz1KxZk0qVKln8rldffZWQkBCz5cUXXyx0zFWqVGHdunX88ssvNGnShOHDhzN06FBTcrcXNn9GWV7ccw9UqABXr6pHkG3bWth59Gh47z31UPPjj7MGjRVCiH8NGjQoR4Oa+++/n/vvvz/X/R966CEeeuihPBsmtWrVioMHD+b7vWfymRqwY8eOuXbh2Lp1a677d+jQgV9yTNyb/3GlSe4oS4mTE9x3n1rPt/rV0xNeeUWtv/YaJCWVaGxCCCHyJomyFBX4OSXAsGEQEQGXLqm+JUIIIWxCEmUp6tpVvf7yC1y/ns/Orq4wbZpaf+stuHatJEMTQgiRB0mUpahaNahbV3XJ2ry5AAf06weNGqms+tZbJR2eEEKIXEiiLGWFqn51cgLjKBvvvQcXL5ZYXEKUV7k1PBGOwxrXVxJlKTMmygKPe/7AA9C6Ndy+DdOnl1hcQpQ3xtFoCjonoSibjNe3OLPXSPeQUtahAzg7w6lTaji7mjXzOUCngxkzoFMn+OgjGDcOauQyr6UQolCcnJzw9/cnISEBAE9PzzznNTQYDKSlpZGSkmJXHeGLwlHKkl85NE0jOTmZhIQE/P39TTNNFYUkylLm46OGs9u+XVW/5jOZuNKxo7oV3bhRDUKwZElJhylEuRAcHAxgSpZ50TSN27dv4+HhYdOJm63BUcpS0HL4+/ubrnNRSaK0gagolShjYwuYKEHdVW7cqIa1e/FFuOuuEo1RiPJAp9MREhJC5cqVSbcwZFZ6ejrbt2/n3nvvtbsJqAvLUcpSkHK4uLgU607SSBKlDXTtCi+/DJs2QUaGqorNV2Qk9OkDK1eqg7/9tsTjFKK8cHJysvgfqpOTExkZGbi7u5fp5AKOU5bSLEfZraAuwyIj1ZCuiYmqT2WBvfaamuLnu+9g584Si08IIUQWSZQ2UKjh7LKrVw+GDlXrMTH5jK4uhBDCGiRR2kihu4kYzZihRlc/fFiGthNCiFIgidJGjMPZ/fxzAYazyy4wMGuUnsmT4dw5a4cmhBAiG0mUNhIeDnXqQGammqOyUAYPVn1MkpJgzJiSCE8IIcS/JFHaUJGrX/V6mDdPPez8+mv44QerxyaEEEKRRGlDxurXQjXoMWrcOOtucuRINcSdEEIIq5NEaUMdO6o+lH/9pYa0K7QpU6BKFXXwjBlWjk4IIQRIorQpX1813jkUofoVwNtbzSoC8OabcPKk1WITQgihSKK0sUJNu5Wb3r0hOlr1qYyJAZkySAghrEoSpY0Zn1Mah7MrNJ0OPvwQ3N3VbNBffmnV+IQQoryTRGljzZqBvz/cuAF79xbxJDVqwKRJan3sWHUyIYQQViGJ0sayD2dXpOeURuPHQ926cOmSGjRdCCGEVUiitAPF6iZi5OYGc+eq9blzYd++YsclhBBCEqVdMCbKPXuKWWvauTM8/jgYDDB8uBr2RwghRLFIorQDERFQu7bKa5s2FfNk77wDfn7w66/wv/9ZJT4hhCjPJFHaieho9Vrs0eiCg+H119X6Sy9BfHwxTyiEEOWbJEo70b27ev3hByt0hRw+XDWnvXFDDaBuMBQ7PiGEKK8kUdqJDh3AwwPOn1dTTRaLkxMsWqT6Vq5fD2+/bZUYhRCiPJJEaSfc3VVbHIB166xwwkaN4IMP1PpLL8Hu3VY4qRBClD+SKO1I9upXqxg6FPr1U62E+vWDq1etdGIhhCg/JFHaEWODnl274Pp1K5xQp4P586FWLYiLgyFDZCxYIYQoJEmUdiQiAurVUzeAxRqlJztfX1i+HFxd4dtvs6pjhRBCFIgkSjtjrH61ynNKo6ZNVf9KgBdeUH0shRBCFIgkSjtjTJTr11u5V8ezz8LDD6vpuPr2lYHThRCigCRR2pl27dR8zPHxcOCAFU+s08GCBRAeDqdOwdNPy/NKIYQoAEmUdsbNDbp0UetWrX4FCAiAZcvA2Rm++go+/tjKXyCEEI7H5oly7ty5RERE4O7uTmRkJDt27LC4f2pqKpMmTSI8PBw3Nzdq1qzJwoULSyna0lEizymNWrWCGTPU+nPPwaFDJfAlQgjhOGyaKJcvX86YMWOYNGkS+/fvp3379kRHRxMXF5fnMY899hibNm1iwYIFnDx5ki+//JJ69eqVYtQlz9hN5Oef4cqVEviCcePUl6Sk4PzEEzjdvl0CXyKEEI7Bpoly1qxZDB06lGHDhlG/fn1mz55NWFgY8+bNy3X/9evXs23bNtatW8d9991H9erVadGiBW3atCnlyEtWWBjcdZdqzFOsOSrzotfDZ59BaCi6kydp/NFHJfAlQgjhGJxt9cVpaWns27ePCRMmmG2Piopidx7DrX333Xc0a9aMt956iyVLluDl5cWDDz7Ia6+9hoeHR67HpKamkpqaanqfmJgIQHp6Ounp6UWO33hscc5hSVSUnsOHnVizxkCfPiUwr6S/P7rFi3GKiqLali2kzZ9P+vDh1v+eUlTS16S0OEo5wHHK4ijlAMcpizXKUdBjbZYoL1++TGZmJkFBQWbbg4KCiM9jaqhTp06xc+dO3N3d+eabb7h8+TIxMTFcvXo1z+eUM2fOZOrUqTm2b9y4EU9Pz2KXI9ZqIwOYCwioCLRjzZp01qxZj76E7v3r9OtH/aVLcRkzhl8uXCC+VauS+aJSVFLXpLQ5SjnAccriKOUAxylLccqRnJxcoP10mmabPgIXLlygSpUq7N69m9atW5u2v/766yxZsoQTJ07kOCYqKoodO3YQHx+Pn58fAKtWraJPnz4kJSXleleZ2x1lWFgYly9fxtfXt8jxp6enExsbS9euXXFxcSnyefI+P4SEOJOYqGPXrgyaNy+Zy5SelsY/vXoRvmkTmpsbmevWobVvXyLfVdJK+pqUFkcpBzhOWRylHOA4ZbFGORITEwkMDOTGjRsW84HN7igDAwNxcnLKcfeYkJCQ4y7TKCQkhCpVqpiSJED9+vXRNI1z585Ru3btHMe4ubnh5uaWY7uLi4tVfkmsdZ6c54WuXeHrr2HjRmdK8jHswZgYwtzd0a9di3Pv3rBjh3pIWkaV1DUpbY5SDnCcsjhKOcBxylKcchT0OJs15nF1dSUyMjLHbXNsbGyejXPatm3LhQsXuHXrlmnb77//jl6vp2rVqiUary2UaDeRbDQnJzKXLoW2bdWIPfffD2fOlOyXCiFEGWHTVq9jx47lk08+YeHChRw/fpznn3+euLg4hv/bqGTixIk8+eSTpv0ff/xxKlasyODBgzl27Bjbt29n/PjxDBkyJM/GPGVZt27q9ddfISGhhL/M0xO+/x4aNoSLF1Wy/OefEv5SIYSwfzZNlH379mX27NlMmzaNu+++m+3bt7Nu3TrCw8MBuHjxolmfSm9vb2JjY7l+/TrNmjXjiSee4IEHHuD999+3VRFKVGgo3HOPGmluw4ZS+MKAAPVF1arB779Djx6Q7e5dCCHKI5s9ozSKiYkhJiYm188+/fTTHNvq1avnMK21CiI6GvbvV9WvAwaUwhdWqaKSZbt2sHcvPPKIutN0dS2FLxdCCPtj8yHshGXG55QbNqh5KktFvXqwdq2qjt24EQYPtvJUJkIIUXZIorRzLVuqGtFr19SQdqX6xV9/rQZQ/+ILNeydzDYihCiHJFHaOWdn1a4GSr71aw7dusGiRWp99mx4661SDkAIIWxPEmUZYBwkvdQTJUD//vDOO2p9wgTIYxxeIYRwVJIoywBjN5H9+1XPjVI3diyMH6/WY2Jg2jSphhVClBuSKMuAypWheXO1vn69jYJ48014+WW1PnmySpil1rpICCFsRxJlGVFao/TkSaeD116DDz9U6/Pnw2OPQUqKjQISQojSIYmyjDA+p9y4UQ2YbjPPPgvLl6t+latWqXrhGzdsGJAQQpSsQifKjIwMnJ2dOXLkSEnEI/LQrBkEBkJiIvz0k42DefRRVQfs4wPbtkGHDjZ6eCqEECWv0InS2dmZ8PBwMuX5VKlycspq1GOz6tfsOnVSSTIoCA4ehDZt1LB3QgjhYIpU9fryyy8zceJErl69au14hAU2f055p3vugd27oWZNNdtI27Zq2DshhHAgRRrr9f333+fPP/8kNDSU8PBwvLy8zD7/7bffrBKcMBcVBXo9HD4M586BXcwsVqMG7Nqlsvhvv6k7za+/zholQQghyrgiJcqHHnrIymGIgqhYEVq1UjdxK1fCmDG2juhfQUGwdSv07g0//gg9e8LcuTBsmGohK4QQZViREuXkyZOtHYcooCeeUIlywQJ47jk7ykM+Pmog9YEDYdkyePpp1UT3f/+DChVsHZ0QQhRZsbqH7Nu3j88//5ylS5eyf/9+a8UkLHj8cXB3hyNH7PBxoKsrLF0Kb7yhBqlduRKaNFF3m0IIUUYVKVEmJCTQuXNnmjdvzujRoxk5ciSRkZF06dKFf/75x9oximz8/aFPH7W+YIFNQ8mdXg//+Y/qw1K7tnqY2rkzTJwIaWm2jk4IIQqtSIly1KhRJCYmcvToUa5evcq1a9c4cuQIiYmJjB492toxijsMHapev/wSkpJsG0uemjVTjXuGDVPjwr7xhmoVK11IhBBlTJES5fr165k3bx7169c3bWvQoAFz5szhhx9+sFpwIncdOkCtWnDzJqxYYetoLPD2ho8/VlWwAQHw66+qS8mCBTKouhCizChSojQYDLi4uOTY7uLigsFgKHZQwjKdDoYMUet2Wf16p0cegUOHVNeR5GR1l/nooyD9cIUQZUCREmXnzp157rnnuHDhgmnb+fPnef755+nSpYvVghN5GzhQPQ7cuRNOnLB1NAVQtSrExqpZSJydVV/Lxo1VdxIhhLBjRUqUH374ITdv3qR69erUrFmTWrVqERERwc2bN/nggw+sHaPIRWgo9Oih1hcutG0sBebkBC++CHv2QJ06cP48dO2qsr40AhNC2KkiJcqwsDB+++031q5dy5gxYxg9ejTr1q1j3759VLWL4WLKB2Ojns8+s/GMIoUVGaka+owcqeqRFy+GevVg0SJ5dimEsDvFmj2ka9eujBo1itGjR3PfffeVRHzCgu7d1aA4CQmwZo2toykkLy/44APVjaRxY/W8csgQ6NixjNQlCyHKC5k9pAxzcYFBg9R6mWjUk5uWLVVr2P/+Fzw9Yft2lTgnT5ZJoYUQdkFmDynjjK1ff/hBPfIrk1xc4IUX4OhRdZucng7TpqlRfbZssXV0QohyrkiJ8v3332fHjh2EhoZSt25dmjZtaraI0lOnDrRvDwYDfPqpraMppurVVR3yV19BcLAanKBzZ3XbLI19hBA2IrOHOIBhw2DHDtX6deJE1W2kzNLpVB/LqCh46SWYN0+1VvrmGxg7Fp5/Hnx9bR2lEKIcKXSizMjIAGDIkCGEhYVZPSBReH36wKhRcOqUGn+8c2dbR2QFfn4wZw4MGAAjRsCBAzBlimoA9J//wLPPqmeaQghRworUmOftt9+Wxjx2xNMT+vVT62W2UU9eWrWCfftg+XKoWxeuXFF9MWvWhA8/hNRUW0cohHBwRaqk69KlC1tl6iS7MmyYev36a7h2zbaxWJ1eD489puYWW7RIPcuMj1e30XXqqL8O/q3pEEIIayvSM8ro6GgmTpzIkSNHiIyMxMvLy+zzBx980CrBiYKLjFS9Kg4dUlNCjhxp64hKgLOzatjz+OMqOU6fDnFx6q+EN95A98oragJpIYSwoiIlyhEjRgAwa9asHJ/pdDqplrUBnU6N1PPccyqHOGSiNHJ1Vc8tBw1SjX1mzoQ//8R54EA6VauGLjVV3YGW6VZNQgh7UeTZQ/JaJEnaTv/+4Oam2r389putoykFHh6qJeypUzB9OpqfH75xcTj366em8/rmGxkSTwhRbIVKlN27d+fGjRum96+//jrXr183vb9y5QoNGjSwWnCicCpUgIcfVuuffGLbWEqVjw9MmkTGH39wom9fNF9fVQfduzc0bQrffisJUwhRZIVKlBs2bCA1WyvDN99802x0noyMDE6ePGm96EShGQdK/+ILNfVjueLvz8l+/cj4/XeYNElNHH3gADz0EDRvrgYzkIQphCikQiVK7Y7/ZO58L2yvc2fVKPTGDdUCtlyqUEE19Dl9GiZMUAOw79sHDzygxpb94QdJmEKIApPWDg5Gr88a/9Xh+lQWVmCgauhz+rTqe+npCXv3qvFkW7WCVatAnqkLIfJRqESp0+nQ6XQ5tgn7MmiQagW7bRv88Yeto7EDlSrBm2+qhDlunGoE9Msv8Mgjah7M+fPh9m1bRymEsFOF6h6iaRqDBg3Czc0NgJSUFIYPH27qR5kqo6TYhbAw6NZN1TCOH68af8rfM0DlyvD22+qH8sEHMHcu/Pmn6mry6qtqAIOYGKhY0daRCiHsSKHuKAcOHEjlypXx8/PDz8+P/v37ExoaanpfuXJlnnzyyZKKVRTCzJlq9qpvv4XFi20djZ0JCsoarGD2bAgPV7OTvPoqVKumEubp07aOUghhJwp1R7lo0aKSikNYWZMmMHWqmoBj9Gjo2FHlA5GNt7caoeHZZ2HFCjV59P79agzZuXPVLCbPPw8tWsgtuRDlmM0b88ydO5eIiAjc3d2JjIxkx44dBTpu165dODs7c/fdd5dsgGXY+PHQujUkJsLgwWrOSpELZ2c1qvy+fRAbq6b4MhjUQOytWqlmxKNHw+bNalJpIUS5YtNEuXz5csaMGcOkSZPYv38/7du3Jzo6mri4OIvH3bhxgyeffJIuXbqUUqRlk7OzmsrR0xO2bFE3SsICnQ7uuw82bFB3lv37qx9eXJx6ptmli6q2HThQPfhNSrJ1xEKIUmDTRDlr1iyGDh3KsGHDqF+/PrNnzyYsLIx58+ZZPO6ZZ57h8ccfp3Xr1qUUadlVu7ZqvwJqGscTJ2wbT5lx992wZAlcvgzffaf63AQGqqlZFi9Wo/4EBsKDD6oZs//5x9YRCyFKSJEGRbeGtLQ09u3bx4QJE8y2R0VFsXv37jyPW7RoEX/99Reff/4506dPz/d7UlNTzVrjJiYmApCenk56MarRjMcW5xylZehQWL3aiY0b9fTvb2D79kxcXNRnZakc+SmRsjg7qybE3brBnDnofvoJ3Xffof/uO3SnTsH338P336PpdGitWqH16IGhRw9o0KDIzzXlmtgfRykHOE5ZrFGOgh6r02w0vM6FCxeoUqUKu3btok2bNqbtM2bM4LPPPst1KLw//viDdu3asWPHDurUqcOUKVNYvXo1Bw4cyPN7pkyZwtSpU3Ns/+KLL/D09LRKWcqCK1fcGT26E0lJrvTrd5y+fX+3dUhlm6bhExdHyM8/E7JnD/6nTpl9nBQURHzz5sQ3b86VBg3QjH+ZCCHsRnJyMo8//jg3btzA19c3z/1sdkdpdOeABZqm5TqIQWZmJo8//jhTp06lTp06BT7/xIkTGTt2rOl9YmIiYWFhREVFWfzB5Cc9PZ3Y2Fi6du2KSxn5T9DZWcfAgbBiRT1Gj65NZKRWJsuRF5uU5d8p59LPnUO/bh26tWvRbd6M16VL1Fyzhppr1qD5+qJFRWHo0QOtW7d8+2nKNbE/jlIOcJyyWKMcxhrG/NgsUQYGBuLk5ER8fLzZ9oSEBIKCgnLsf/PmTX799Vf279/PyH8nWzQYDGiahrOzMxs3bqRz5845jnNzczMNkJCdi4uLVX5JrHWe0jBggBoXfMUKHUOGOLNvH6Yq2LJUjvzYpCwREaqbybPPqkY+P/6onm2uXYvu0iV0K1eiX7lSVcc2baoaBnXuDO3aqbFo7aUcJcRRyuIo5QDHKUtxylHQ42zWmMfV1ZXIyEhiY2PNtsfGxppVxRr5+vpy+PBhDhw4YFqGDx9O3bp1OXDgAC1btiyt0MssnU51DwwOhuPH1QQbogR4eUGvXmqw3QsXYM8e9cNu3FgNxr5vH7z1lnruGRAA7dvDlCmwfTvI6FZC2B2bVr2OHTuWAQMG0KxZM1q3bs1HH31EXFwcw4cPB1S16fnz51m8eDF6vZ5GjRqZHV+5cmXc3d1zbBd5CwxUc1X27Anvvgvdu0tH+hKl16sZS1q2VKMBXbig+ups3gybNsHff8POnWqZOhU8PHBq25baISHofHzUcXnccQohSodNE2Xfvn25cuUK06ZN4+LFizRq1Ih169YR/u8QMhcvXsy3T6UovB494Kmn4OOPYdgwJ2bOtPmj6vIjNBSeeEItmqaGyjMmzc2bISEB/Y8/0gBU9xQnJ3Un2rKlGvygVSvV50dv87FChCg3bP4/ZExMDDExMbl+9umnn1o8dsqUKUyZMsX6QZUD77yjHqOdPq1jwYJG9Olj64jKIZ0OatRQy7BhKnEeO0bmxo1c+uorQuLi0F24oAY/2L9fzXICqrrWeJfaqhW0aQPFaJgmhLDM5olS2IaPjxq1p0MHjU2bwlm1KoO+fW0dVTmn00HDhhjq1GFvjRp0794dl0uX1DNO47Jvnxr0YP16tYC6u4yMVAP6duyoGghJ4hTCaiRRlmPt28PYsQbeeceJoUOdqF9f1fIJO1K1KvTpg+mWPz0dDh3KSpy7dqnq27171fLf/2Ylzg4dshKnn59NiyFEWSYPOsq5adMMNG78D0lJOh54ABISbB2RsMjFRSXBZ59VzzBPnVJj0S5ZooZgqllTDei+d68au7BnT6hQAZo3hxdfVOPYyhi1QhSKJMpyzsUFXnxxL7VqacTFqSFMpYdCGRMWpgZw/+QTNRG1MXEOGwa1aqnE+euv6m4ze5eUyZNh2za54ELkQxKlwNs7nVWrMvDzUzV5I0aodiWijDImzo8/hj/+gLNnVeIcPFhNTJ2errqjTJumqmYDAqBrV3jjDfjlF5lKTIg7yDNKAUC9evDVVxAdDYsWQcOGMG6craMSVlG1qkqc/fubd0kxLpcuqSbQP/6o9nd1VQ+rmzZV1bxNm0KjRuDubttyCGEjkiiFSVSUGoTguefUpM/160P37raOSlhVHl1STElz61a4fl1V1f76a9Zxzs4qWTZtmrU0aqSaTwvh4CRRCjOjRsHRo/DRR/B//wc//aTuLoWD+rdLCg0bqotvMKg7zn374Lff1LJvH1y9CgcOqGXhwqzjq1RR1RH16kHdulnrVasWeZoxIeyNJEphRqeDDz6AkydVO48HH4Sff1ZD34lyQK9XLWdr1oTHHlPbNE01EDImTuMSHw/nz6tl0ybz83h5QZ06ONWpQz2DAV18vLqLDQ9Xz0mlGleUIZIoRQ6urvD119Cihep90KcPbNyototySKdTCS48HB5+OGv7tWvqL6oTJ9RiXP/zT9UFZf9+9Pv3UxdgxQrzcwYFZZ3TuNSooe5Gw8PV0H1C2AlJlCJXFSuqWaJat1Z3liNHwv/+J7VpIpuAgKzxZ7NLT1fVtydOkHn0KHHbtxMO6OPi1CDwSUmqAdGlS6qV7Z3c3KBOnaxqXONSpw54e5dK0YTIThKlyFPDhrBsmeqz/vHHqu3G6NG2jkrYPRcXldTq1MEQHc2hBg2o2r07ehcXVY179apKmMYlLg7OnFF3or//rvp1Hj6sljuFhalWZo0aqeWuu6BBA/D0LPViivJDEqWwqHt31U/9hRfg+edV/3VpCSuKTKdT1RUVK6qWs3fKzFTJ01idm71KNyFB9Qk9e1Y9C8h+zpo1sxKn8bV2bdVaV4hikt8ika+xY1VL2EWL1COqr79Wd5lCWJ2TU1b3lTv/Irt6VSXMY8fU3eaRI+r1n3/U3eiff8Lq1Vn7u7mphHn33XDPPWpp3Fjm9xSFJolS5EunUzM8JSaqJPnww7B8uRruTohSU6GCmlKsTRvz7QkJWVW1xuR59Kh6Fnpnf1CdTlULZ0+eDRqoc3t4yEN4kStJlKJAXF3V88oBA9TrY4/B55+rvpZC2FTlytCli1qMDAbVZPvAgaz5PPfvV11aTp5Uy/Ll5udxcVGzrPj753jV+/hQ5/JldNeuqURbo4ZquSuJtVyQRCkKzNlZJUdXV1i8GJ54AtLS4MknbR2ZEHfQ69UD9Vq1MJuVPD4+K3kaX//6SyXW9HS4fFktd3AC6gMsXZq10cMDIiJU0jS+GtcjIqSFrgORRCkKxclJPat0dVWTVQwapP5/GTrU1pEJUQDBwWoGlW7dsrYZDHDrFty4oYbvu3HDfP36dTKvXuXcgQOEZWaiP31aNSi6fVs9Lz12LPfvCgyE6tWzEqdxvXp1tcigC2WGJEpRaHq96lPp5gZz5qghQ1NTISbG1pEJUQR6Pfj6qiUsLNddDOnpHFi3jlBjN5e0NNWt5fRpVcV76pT5+rVrWXen2Z+RZhcUpJ6NGr/bxyf3dV9fqFRJ7R8crFoMy4AMpUoSpSgSvV4NdefmBrNmqXmEU1NVFxIhHJ6ra1bVbm5u3FB9Q0+fznrNvn7rVtagC4Wl16vnssbEaem1QgW1vyO5ehX++gvdyZO4ZWaWyldKohRFptPB22+rZDlzpupGkpoKEybYOjIhbMzPD5o0UcudNA2uXFF3pDduqObkN2+q1+zrxtcbN1TL3kuX1B2qwaCetcbHw8GDluNwdlZJNVsC1QcGUvviRfS//64GanB3V/+I3dxyrru7q308PdUzWU9P9Vl+jZg0Td11374Nycnq9fZtyMhQd8NOTiq2O9eNr0lJ6tlxbsv166poQMDEiUW6PIUliVIUi04Hr7+u/u1MmQITJ6p/H6+8Ig0ChciVTqeeXxZlpoGMDNVvND5eJc7sr8Z14/urV9X+Fy6o5V9OQIPixu/hkZU4jd1qsifE5OSSnf09JARDjRpkltIA1JIoRbHpdDB5sqqNeukltZ6QoEb08fCwdXRCOBBnZwgJUUt+0tLUP8Q7kmrmhQucO3GCsEqV0Kenq2qg1FRIScm5nj3xZWSo82qaep+crO6M86PXZyVWFxc1+lJGhno1Lsb3BkNWOatXz5rJJvtSowZ4epKZns4/69YV+UdZGJIohdVMnKjuLMeNU418tm5Vrelzq30SQpQwV1c1L2jVqmabczRMKqiMDPOq1OyvmpZ1d3nn3aaLS8GrlzRNJUydzq4aLEmiFFY1dqwas3rwYDU4SosWqmp27FjHa1MgRLni7Kxa4/r4lNx36HR2OT6v/NclrC46Wo0i1quXqv0ZPx7uu091PRNCiLJGEqUoEZUqwTffwEcfqRqYLVvUeNR3jhomhBD2ThKlKDE6HTz1lBoprEUL1ar7//5PDXl344atoxNCiIKRRClKXO3asHOn6jKi18OSJaqBz44dto5MCCHyJ4lSlAoXF5g2TSXHiAg1N2+HDur5ZVKSraMTQoi8SaIUpapNGzWYyODBqiX4229Dw4awZo2tIxNCiNxJohSlzscHFi6E77+HatXU3eUDD6iJoM+ds3V0QghhThKlsJmePdUMRS++qLpOffON6oP57rtZg4AIIYStSaIUNuXlBW++Cb/9pqplb91SgxM0bw6//GLr6IQQQhKlsBN33aUa+nz8MQQEqC4lrVqp6bv+nSxACCFsQhKlsBt6vZoE+sQJ1ddS02DuXFUd+9FH6m5TCCFKmyRKYXcqV4bPPoPNm6FuXTXpwTPPqAkTnnoK9uwp2Rl8hBAiO0mUwm516qS6krzzjhq04NYt+OQTaN1aVdW++66ax1YIIUqSJEph19zcVOOekydh2zZVJevhoWYmGTsWQkPhscdgwwY1O48QQlibJEpRJuh0cO+9qkr24kWYNw+aNYP0dFixArp1g7p1nVm6tB6//SZVs0II65FEKcocPz8YPhz27lWtY0eNUi1l4+J0rFhRl1atXAgPV9s3b1bJVAghisrmiXLu3LlERETg7u5OZGQkOyyMlL1q1Sq6du1KpUqV8PX1pXXr1mzYsKEUoxX2pkkTeP99uHABlizJoE2b83h5aZw9Cx9+CF26QFCQqrL95hsZV1YIUXg2TZTLly9nzJgxTJo0if3799O+fXuio6OJi4vLdf/t27fTtWtX1q1bx759++jUqRMPPPAA+/fvL+XIhb1xd4e+fTVefPFXLlzI4PvvYcgQCAyEa9fUjCW9e6v3vXqpqtuff4bkZFtHLoSwd862/PJZs2YxdOhQhg0bBsDs2bPZsGED8+bNY+bMmTn2nz17ttn7GTNm8O233/L9999zzz335PodqamppKammt4nJiYCkJ6eTnox6uSMxxbnHPbAUcoBWWVwdk7n/vvh/vthzhz46Scd336r49tv9Zw5o+O77+C779QxOp1G7drQpIlmWho31ggOVs9FbVkOR7omZb0sjlIOcJyyWKMcBT1Wp2m2afaQlpaGp6cnK1as4OGHHzZtf+655zhw4ADbtm3L9xwGg4Hq1avz4osvMnLkyFz3mTJlClOnTs2x/YsvvsDT07PoBRBljqbB33/7smdPCCdPBnDqlB83brjnuq+fXwoREYnUrXuVLl3iqFz5dilHK4QoacnJyTz++OPcuHEDX1/fPPezWaK8cOECVapUYdeuXbRp08a0fcaMGXz22WecPHky33P897//5Y033uD48eNUrlw5131yu6MMCwvj8uXLFn8w+UlPTyc2NpauXbvi4uJS5PPYmqOUA4pWlvh4OHhQx8GDOg4dUq+//w6alnU7qddrdO+uMWKEgS5dNPQl/MCivF8Te+Qo5QDHKYs1ypGYmEhgYGC+idKmVa8AujvqtzRNy7EtN19++SVTpkzh22+/zTNJAri5ueHm5pZju4uLi1V+Sax1HltzlHJA4coSFqaWnj2ztiUnw5EjsH8/fPUVbN6sY80aHWvW6KldG0aMgEGDVEvbklRer4k9c5RygOOUpTjlKOhxNmvMExgYiJOTE/Hx8WbbExISCAoKsnjs8uXLGTp0KF999RX33XdfSYYpyiFPT2jRQg2bt2mTmgps1Cjw9YU//lADHVSposallXZkQjg+myVKV1dXIiMjiY2NNdseGxtrVhV7py+//JJBgwbxxRdf0KNHj5IOUwjq11ddUM6fh/nz1fB5t2/DggXQtKkaUm/BApVQZXQgIRyPTatex44dy4ABA2jWrBmtW7fmo48+Ii4ujuHDhwMwceJEzp8/z+LFiwGVJJ988knee+89WrVqZbob9fDwwM/Pz2blEOWDt7e6y3z6adi1S81ssnKlGqR9zx61j4eH6tt5zz1ZS6NGqvuKEKJssmmi7Nu3L1euXGHatGlcvHiRRo0asW7dOsLDwwG4ePGiWZ/K//3vf2RkZPDss8/y7LPPmrYPHDiQTz/9tLTDF+WUTgft2qll1ix1N7l2rRrAPTnZPHECODlBgwYqaTZrBt27Q82atotfCFE4Nm/MExMTQ0xMTK6f3Zn8tm7dWvIBCVEIwcEwaZJaMjPVM8z9+82XK1fg8GG1LF4Mo0er6tuHH1ZLkya267MphMifzROlEI7CyQnq1VNLv35qm6bBuXNZSXPbNti+PStxTpsG1aurhPnQQ9C2rS1LIITIjSRKIUqQTpfVBeXBB9W2K1dgzRpYvVpND3bmjJpb8913oVIl6NnTidDQIDp0AH9/GwYvhADsYFB0IcqbihVh4EA1SPs//8CqVTBggEqK//wDixbpef31VlSu7EzHjjB9unrmmZFh68iFKJ8kUQphQ15eqtp18WJISIAff4QRIzKpVCmZ9HQd27bBK6+oLigVK6oB3T/4AI4flzk3hSgtUvUqhJ1wcVHTgt17r4GoqFjq1u3Otm0u/Pijmlfz2jXMBnQPDYVOnVS1boUKaqQg42v2dW9vaSwkRHFIohTCDul0UKuWGuxg+HDVonb/fnXH+eOPsHOnmoNz6dL8z+XsrO5GmzWD++5TS8OGkjyFKChJlEKUAU5OKtE1awYTJqiRgXbtgp9+gsuX1d3m1avqNft6Wpp6tnnpkurruXatOl9QkEqYXbqo17Aw25ZPCHsmiVKIMsjDI+vuMC+aphLq1atw8aLqmvLjj6p7yqVL6m7UeEdap446V+fO4OcHN2+q5dYt81fj+q1bqlo3IsJ8qVpV3cEK4UjkV1oIB6XTqQHePT1VAmveHF54AVJT1Z2osRp37174/Xe1zJ1bvO90dlZ3pxERUKMGVKumJy0tiPbt1TNTIcoiSZRClDNubtCxI6auJ9evw9atKmnu2KHuRL29wccn6zX7ure3aq17+TKcPp21/P23quo1vt+8GcAJaMUbb2i0aKHuWLt0Ua14c5n9Tgi7JIlSiHLO31+NCvTQQ8U7j8GgGhhlT56//25g06bbXLrkxe7dsHu3Ss7u7mqs3C5d1NK0qXoOK4Q9kkQphLAKvV5V8VatCu3bq23p6ZmsW/cjDRp0Z8cOFzZtUnea8fFZVb+gnovWqqVa92ZkqCX7evZtvr5w990quRpnaAkNlVa8ouRIohRClLjq1aF2bRgyRFXtHj+uJsXetElV+964Afv2FexcV6+qYf9Wr87aVrmy+dRmTZuqZ6R6GVJFWIEkSiFEqdLp1LRjDRrAqFFZfUQvXVKNgbIvTk4531+6pPb/7Tf1evy4GtVowwa1GLm4qGplf391x2ppvWJFtVSooF79/SXJiiySKIUQNmXsI1pQ9eurhkhGyclqJpbsU5sdOqRa9/7zj1oKS6/PGt1IJVAnkpObsmWLnkqV1LbAwKwEa1yXBkqOSRKlEKJM8/SEli3VYpServqO3rihWvUaX7OvG1+NAzRcuaKWW7dUwyTj+z/+ADUsdhj5TYnr5aUGc4iMhDZt1HL33eDqWhIlF6VFEqUQwuG4uEC1akU7NjVVJU9jorx6FRISMti16ySVKtXj2jUn02dXrqhuMlevqirkpCQ4dUotK1ao87m7qz6sxsTZurWaTi0vycnqnMbzX7umBpi4s8rYx0caMJUWSZRCCJGNmxsEB6vFKD1dIyjoT7p3r4OLS85+LAYDJCaqxBYXBz//jKk7zJUrqn/qjh1Z+9euraqbMzLME+6VK5CSUrA49XqVMLMn0AYN1OTfbduqPxQkkVqHJEohhCgmvT4rWdWsqWZ1AdXC9/ffs5Lm7t1w7JiqzlVVurlzccl67hkQoJKnsbr42jVVtWwwZI3ta7R1a9boSlWqqITZrp16bdw4/+EFjedMSFCLk5OqSq5cWXXLKa+JVxKlEEKUEJ0O6tZVy+DBatu1a2oi7oMH1fPV7I2BjK+WpkbTNPPEaXz2evmy6mKza5dq0HT+PHz1lVpAnbNlS2jVSs/Zs7XYtk3PP/9kJcVLl1TDp7wmCHdzUwmzcuWs5Gl8dXVVd9SJiWo8YOP6ne8NBtVAKvu0cLm9DwpS4w/7+Fj5ghSRJEohhChFAQEQHa2WotDp1DNLDw8ICTH/rH9/9ZqUBL/8opKmcZaZGzeMfVedgIYWv8PfXz1HzcxUSfTWLfXs9uxZtRRHYVohh4ZCvXo5lypVihdDYUmiFEIIB+Plpap/jVXABgMcPaqS5u7dBs6ePU/TpqEEBzvluEusVClnN5fkZPM7z+zrly5ljZhkXHx8cn8P5lPBZV+ybzt3Tp3/wgW1qHGDs3h6Qp06zvToEUT37iX/85REKYQQDk6vh7vuUsvQoZmsW/cb3bsH59owKTeenmp0perVSzRMM9euwcmTajlxImv580+VuA8c0NGjR+nEIolSCCGE3QkIgFat1JJderrqfnP0aAZJSddyP9jKZJAmIYQQZYaLi2oc9cADGr6+aaXynZIohRBCCAskUQohhBAWSKIUQgghLJBEKYQQQlggiVIIIYSwQBKlEEIIYYEkSiGEEMKCcjfggKZpACQmJhbrPOnp6SQnJ5OYmIiLi4s1QrMJRykHOE5ZHKUc4DhlcZRygOOUxRrlMOYBY17IS7lLlDdv3gQgLCzMxpEIIYSwBzdv3sTPzy/Pz3VafqnUwRgMBi5cuICPjw+6YkyulpiYSFhYGGfPnsXXONpvGeQo5QDHKYujlAMcpyyOUg5wnLJYoxyapnHz5k1CQ0PR6/N+Elnu7ij1ej1Vq1a12vl8fX3L9C+bkaOUAxynLI5SDnCcsjhKOcBxylLccli6kzSSxjxCCCGEBZIohRBCCAskURaRm5sbkydPxu3OGU7LGEcpBzhOWRylHOA4ZXGUcoDjlKU0y1HuGvMIIYQQhSF3lEIIIYQFkiiFEEIICyRRCiGEEBZIohRCCCEskERZBHPnziUiIgJ3d3ciIyPZsWOHrUPK15QpU9DpdGZLcHCw6XNN05gyZQqhoaF4eHjQsWNHjh49asOIle3bt/PAAw8QGhqKTqdj9erVZp8XJO7U1FRGjRpFYGAgXl5ePPjgg5w7d64US6HkV5ZBgwbluEatWrUy28ceyjJz5kyaN2+Oj48PlStX5qGHHuLkyZNm+5SF61KQcpSVazJv3jwaN25s6nzfunVrfvjhB9PnZeF6FKQctroekigLafny5YwZM4ZJkyaxf/9+2rdvT3R0NHFxcbYOLV8NGzbk4sWLpuXw4cOmz9566y1mzZrFhx9+yN69ewkODqZr166msXFtJSkpiSZNmvDhhx/m+nlB4h4zZgzffPMNy5YtY+fOndy6dYuePXuSmZlZWsUA8i8LQLdu3cyu0bp168w+t4eybNu2jWeffZY9e/YQGxtLRkYGUVFRJCUlmfYpC9elIOWAsnFNqlatyhtvvMGvv/7Kr7/+SufOnenVq5cpGZaF61GQcoCNrocmCqVFixba8OHDzbbVq1dPmzBhgo0iKpjJkydrTZo0yfUzg8GgBQcHa2+88YZpW0pKiubn56fNnz+/lCLMH6B98803pvcFifv69euai4uLtmzZMtM+58+f1/R6vbZ+/fpSi/1Od5ZF0zRt4MCBWq9evfI8xl7LkpCQoAHatm3bNE0ru9flznJoWtm9JpqmaQEBAdonn3xSZq+HkbEcmma76yF3lIWQlpbGvn37iIqKMtseFRXF7t27bRRVwf3xxx+EhoYSERHB//3f/3Hq1CkATp8+TXx8vFm53Nzc6NChg12XqyBx79u3j/T0dLN9QkNDadSokV2WbevWrVSuXJk6derw1FNPkZCQYPrMXsty48YNACpUqACU3etyZzmMyto1yczMZNmyZSQlJdG6desyez3uLIeRLa5HuRsUvTguX75MZmYmQUFBZtuDgoKIj4+3UVQF07JlSxYvXkydOnW4dOkS06dPp02bNhw9etQUe27l+vvvv20RboEUJO74+HhcXV0JCAjIsY+9XbPo6GgeffRRwsPDOX36NK+88gqdO3dm3759uLm52WVZNE1j7NixtGvXjkaNGgFl87rkVg4oW9fk8OHDtG7dmpSUFLy9vfnmm29o0KCBKUGUleuRVznAdtdDEmUR3Dk9l6ZpxZqyqzRER0eb1u+66y5at25NzZo1+eyzz0wPw8tiuaBocdtj2fr27Wtab9SoEc2aNSM8PJy1a9fSu3fvPI+zZVlGjhzJoUOH2LlzZ47PytJ1yascZema1K1blwMHDnD9+nW+/vprBg4cyLZt20yfl5XrkVc5GjRoYLPrIVWvhRAYGIiTk1OOv0wSEhJy/LVm77y8vLjrrrv4448/TK1fy1q5ChJ3cHAwaWlpXLt2Lc997FVISAjh4eH88ccfgP2VZdSoUXz33Xds2bLFbOq6snZd8ipHbuz5mri6ulKrVi2aNWvGzJkzadKkCe+9916Zux55lSM3pXU9JFEWgqurK5GRkcTGxpptj42NpU2bNjaKqmhSU1M5fvw4ISEhREREEBwcbFautLQ0tm3bZtflKkjckZGRuLi4mO1z8eJFjhw5YtdlA7hy5Qpnz54lJCQEsJ+yaJrGyJEjWbVqFZs3byYiIsLs87JyXfIrR27s9ZrkRtM0UlNTy8z1yIuxHLkptetR5GZA5dSyZcs0FxcXbcGCBdqxY8e0MWPGaF5eXtqZM2dsHZpF48aN07Zu3aqdOnVK27Nnj9azZ0/Nx8fHFPcbb7yh+fn5aatWrdIOHz6s9evXTwsJCdESExNtGvfNmze1/fv3a/v379cAbdasWdr+/fu1v//+u8BxDx8+XKtatar2448/ar/99pvWuXNnrUmTJlpGRobdlOXmzZvauHHjtN27d2unT5/WtmzZorVu3VqrUqWK3ZVlxIgRmp+fn7Z161bt4sWLpiU5Odm0T1m4LvmVoyxdk4kTJ2rbt2/XTp8+rR06dEh76aWXNL1er23cuFHTtLJxPfIrhy2vhyTKIpgzZ44WHh6uubq6ak2bNjVrTm6v+vbtq4WEhGguLi5aaGio1rt3b+3o0aOmzw0GgzZ58mQtODhYc3Nz0+69917t8OHDNoxY2bJliwbkWAYOHKhpWsHivn37tjZy5EitQoUKmoeHh9azZ08tLi7OrsqSnJysRUVFaZUqVdJcXFy0atWqaQMHDswRpz2UJbcyANqiRYtM+5SF65JfOcrSNRkyZIjp/6RKlSppXbp0MSVJTSsb1yO/ctjyesg0W0IIIYQF8oxSCCGEsEASpRBCCGGBJEohhBDCAkmUQgghhAWSKIUQQggLJFEKIYQQFkiiFEIIISyQRCmEEEJYIIlSCFFgOp2O1atX2zoMIUqVJEohyohBgwah0+lyLN26dbN1aEI4NJmPUogypFu3bixatMhsm5ubm42iEaJ8kDtKIcoQNzc3goODzRbjbO46nY558+YRHR2Nh4cHERERrFixwuz4w4cP07lzZzw8PKhYsSJPP/00t27dMttn4cKFNGzYEDc3N0JCQhg5cqTZ55cvX+bhhx/G09OT2rVr891335VsoYWwMUmUQjiQV155hUceeYSDBw/Sv39/+vXrx/HjxwFITk6mW7duBAQEsHfvXlasWMGPP/5olgjnzZvHs88+y9NPP83hw4f57rvvqFWrltl3TJ06lccee4xDhw7RvXt3nnjiCa5evVqq5RSiVBVr7hEhRKkZOHCg5uTkpHl5eZkt06ZN0zRNTRs1fPhws2NatmypjRgxQtM0Tfvoo4+0gIAA7datW6bP165dq+n1ei0+Pl7TNE0LDQ3VJk2alGcMgPbyyy+b3t+6dUvT6XTaDz/8YLVyCmFv5BmlEGVIp06dmDdvntm2ChUqmNZbt25t9lnr1q05cOAAAMePH6dJkyZ4eXmZPm/bti0Gg4GTJ0+i0+m4cOECXbp0sRhD48aNTeteXl74+PiQkJBQ1CIJYfckUQpRhnh5eeWoCs2PTqcDQNM003pu+3h4eBTofC4uLjmONRgMhYpJiLJEnlEK4UD27NmT4329evUAaNCgAQcOHCApKcn0+a5du9Dr9dSpUwcfHx+qV6/Opk2bSjVmIeyd3FEKUYakpqYSHx9vts3Z2ZnAwEAAVqxYQbNmzWjXrh1Lly7ll19+YcGCBQA88cQTTJ48mYEDBzJlyhT++ecfRo0axYABAwgKCgJgypQpDB8+nMqVKxMdHc3NmzfZtWsXo0aNKt2CCmFHJFEKUYasX7+ekJAQs21169blxIkTgGqRumzZMmJiYggODmbp0qU0aNAAAE9PTzZs2MBzzz1H8+bN8fT05JFHHmHWrFmmcw0cOJCUlBTeffddXnjhBQIDA+nTp0/pFVAIO6TTNE2zdRBCiOLT6XR88803PPTQQ7YORQiHIs8ohRBCCAskUQohhBAWyDNKIRyEPEURomTIHaUQQghhgSRKIYQQwgJJlEIIIYQFkiiFEEIICyRRCiGEEBZIohRCCCEskEQphBBCWCCJUgghhLDg/wF+1Q7zSNZ94wAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 500x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# 创建图形\n",
    "plt.figure(figsize=(5, 3))\n",
    "\n",
    "plt.plot(x_axis_epoch, y_axis_train_error, label='Train Error', color='blue')\n",
    "plt.plot(x_axis_epoch, y_axis_test_error, label='Test Error', color='red')\n",
    "\n",
    "# 添加标题和标签\n",
    "plt.title('Train Error and Test Error')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Error')\n",
    "\n",
    "# 添加图例\n",
    "plt.grid(True)\n",
    "plt.legend()\n",
    "\n",
    "# 显示图形\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "myenv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
